fix(bench): clear compiler caches before every cold build trial #1154

Context

The nightly Arduino CLI vs PlatformIO vs fbuild benchmark currently reports an implausible fbuild cold median. Its first production run measured:

• fbuild cold: 79.787 ms
• fbuild warm: 51.224 ms

Before each fbuild cold trial, the runner calls:

fbuild clean all bench/blink --environment uno --release

clean all intentionally removes the selected project outputs and matching reusable framework artifacts, but it does not clear the embedded zccache object store at fbuild_paths::get_fbuild_root().join(zccache). The “cold” build therefore receives compiler-cache hits.

Arduino CLI currently removes only its build directory. PlatformIO runs its project clean target. Neither tool’s download/HTTP cache cleaner is invoked.

Proposal

Public fbuild operation

Add:

fbuild clean cache [PROJECT_DIR] [-e ENV] [--quick|--release]

The operation must:

1. Refuse safely if another daemon operation is active.
2. Gracefully stop the daemon so the embedded zccache store has no live owners.
3. Idempotently remove only the mode-correct compiler cache root returned through fbuild_paths; never hard-code dev, prod, or a home-directory path.
4. Restart the daemon even when cleanup encounters an error, returning the relevant failure afterward.
5. Perform the existing clean all behavior for the selected project/environment/profile.
6. Retain installed packages, platforms, frameworks, and toolchains.

Because the compiler cache is global to the active fbuild mode/trust domain, the command and documentation must explicitly describe that scope.

Symmetric benchmark preparation

Before every cold trial, outside the timed interval:

• Arduino CLI:
  • run arduino-cli cache clean;
  • remove the configured build directory.
• PlatformIO:
  • run pio system prune --cache --force;
  • run pio run --target clean for the Uno environment.
• fbuild:
  • run fbuild clean cache ....

The warm measurement must immediately repeat the build without another cleanup.

Installed toolchains/framework packages remain primed so the benchmark measures compilation rather than installation or network latency.

Published metadata

Update the site and JSON cold definition to state that project outputs, reusable framework objects, compiler-object caches, and download/HTTP caches are cleared while installed packages/toolchains remain.

Acceptance criteria

• fbuild clean cache [PROJECT_DIR] [-e ENV] [--quick|--release] parses and is documented.
• The command clears the active mode’s entire embedded zccache artifact store.
• It also removes the selected project outputs and matching framework-cache entry.
• Cleanup is idempotent.
• The daemon is never deleted from underneath a running operation.
• The daemon is usable again after successful cleanup.
• A cleanup failure still attempts to restore daemon availability.
• Tests use temporary directories and prove the compiler-cache root is removed without deleting sibling fbuild roots.
• Benchmark tests prove each tool’s complete cleanup sequence occurs before every cold build and not before the warm build.
• Arduino and PlatformIO download/HTTP caches are cleaned without uninstalling packages.
• Cleanup duration is excluded from timing.
• Published HTML/JSON accurately describes the stronger cold-cache contract.
• The Rust benchmark source remains below the repository’s 1,000-line limit, splitting tests into an existing-directory sibling module if needed.
• After merge, a production workflow run publishes three-trial results and its log proves cache cleanup ran before every cold trial.

Open questions

None. The cold-cache contract was confirmed by the requester.

Related issues

• feat(build): retain verified framework caches for clean builds #1089 — introduced clean sketch|all and intentionally scoped all to selected project/framework artifacts rather than the global compiler-object cache.