Skip to content

feat(bench): publish ESP32-S3 build timings alongside Arduino Uno #1488

Closed

Description

@zackees

Context

The nightly build-comparison workflow currently benchmarks only the Arduino Uno Blink fixture. Its published benchmark.svg heading/description name Arduino Uno, and latest.json exposes one result set containing Arduino CLI, PlatformIO, and fbuild timings.

Consumers need the same cold/warm comparison for ESP32-S3 in the existing stable artifact at:

https://raw.githubusercontent.com/FastLED/fbuild/benchmark-stats/benchmark.svg

Proposal

Extend the existing Blink benchmark with an ESP32-S3 PlatformIO environment and make the benchmark runner describe a board by its display name, Arduino FQBN, PlatformIO/fbuild environment, and toolchain pins. Run the same three-tool cold/warm measurement for Arduino Uno and ESP32-S3, then render both board groups into the existing SVG and machine-readable artifacts.

Keep one workflow, one fixture, one benchmark runner, and one published benchmark-stats branch.

Acceptance criteria

  • RED: add a focused runner test proving the current SVG/output model cannot represent both Arduino Uno and ESP32-S3 result groups.
  • GREEN: the focused benchmark-runner tests pass after the output model and renderer support both boards.
  • bench/blink/platformio.ini includes an ESP32-S3 environment usable by PlatformIO and fbuild.
  • The workflow installs/primes both Arduino Uno and ESP32-S3 toolchains outside timed measurements.
  • Each board is measured with Arduino CLI, PlatformIO, and fbuild using the same cold/warm trial policy.
  • The existing benchmark.svg contains two labeled board groups: Arduino Uno and ESP32-S3.
  • The machine-readable latest/history output preserves board identity for both groups.
  • After the workflow publishes, curl -fsSL https://raw.githubusercontent.com/FastLED/fbuild/benchmark-stats/benchmark.svg returns an SVG containing both Arduino Uno and ESP32-S3.

Decisions

  • Use the existing Blink fixture and runner because this is the exact comparison path already published for Uno; duplicating a benchmark implementation would invite drift.
  • Keep the existing artifact URL because downstream curl/image consumers should gain the second board without discovering a new endpoint.
  • Benchmark the same three tools for both boards so the two graph members remain directly comparable.
  • Treat this as a benchmark schema extension rather than a second standalone report so board identity is explicit in SVG, latest JSON, and history.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions