feat(build): retain verified framework caches for clean builds #1089

Context

ESP32 clean-build profiling showed that the dominant work is reusable framework work, not LTO:

• framework built-in libraries: about 5.5 s
• framework core/variant compilation: about 15-16 s in the original profile
• link/convert/size: about 5.5 s

The desired contract is two cleanup scopes:

• sketch: remove only the selected project/environment/profile outputs; verified reusable framework artifacts remain available.
• all: remove those project outputs and the exact reusable framework-cache entries applicable to the selected environment/profile.

Project outputs live under the BuildLayout-resolved project build directory. Reusable framework artifacts live under the global cache returned by fbuild_paths::get_cache_root() / fbuild_packages::Cache; do not invent paths manually.

Current status

PR #1093 (merged as 2c24dddd) completed the first implementation slice:

• fbuild build --clean and fbuild deploy --clean preserve and hydrate the ESP32 framework core cache.
• --clean-all is wired through the CLI, daemon requests, and BuildParams.
• ESP32 --clean-all evicts the exact core and built-in-library cache entries before rebuilding.
• ESP32 built-in-library archives are content-addressed under the global framework-libs cache.
• Cache keys cover profile, compiler/signature inputs, framework source contents, and relevant project headers/overlays.
• ESP32 A/B evidence improved a clean build from 255.0 s (cold populate) to 68.6 s (cache hit), with core compilation dropping from 194.3 s to 2.0 s in that run.

Do not reimplement or replace the #1093 cache formats. Build on them.

Remaining gaps on current main

1. There is no standalone fbuild clean sketch|all command.
2. crates/fbuild-build-engine/src/pipeline/sequential.rs still sets core_cache = None whenever params.clean is true. Therefore normal --clean builds on AVR, Teensy, STM32, ESP8266, and the other shared-sequential orchestrators do not hydrate/store the reusable core cache.
3. The shared sequential pipeline never reads params.clean_all, so it cannot evict its exact FrameworkCoreCache entry.
4. FrameworkCoreCache exposes hydrate and store, but has no idempotent exact-entry remove operation.

Required CLI contract

Add these native-fbuild forms:

fbuild clean sketch [PROJECT_DIR] [-e ENV] [--quick|--release]
fbuild clean all    [PROJECT_DIR] [-e ENV] [--quick|--release]

Rules:

• PROJECT_DIR defaults to ..
• ENV resolves exactly like build: explicit -e, then default_envs, then default.
• Profile selection matches build: --quick and --release conflict; omitted means release.
• clean sketch deletes only the BuildLayout directory for the selected environment/profile.
• clean all performs clean sketch, then removes only the matching reusable core entry and, for ESP32, the matching built-in-framework-library entry.
• Both scopes are idempotent: missing directories/cache entries are success, not errors.
• The command must not compile, link, deploy, flash, hydrate a cache, or populate a cache.
• Do not implement this by calling purge project (too broad: all envs/profiles) or purge all (catastrophically broader than the selected cache entries).
• Do not add --platformio; this command owns native fbuild artifacts only.

Implementation plan

1. Add failing tests first

Start with parser and filesystem/cache tests for the contract below. Keep the new logic testable with tempfile::TempDir; never use literal /project placeholders for path-sensitive cache tests.

2. Finish shared core-cache semantics

Files:

• crates/fbuild-build-engine/src/framework_core_cache.rs
• crates/fbuild-build-engine/src/pipeline/sequential.rs

Changes:

• Add FrameworkCoreCache::remove() -> std::io::Result<()>. It removes only self.path; NotFound is success. Do not remove core_artifacts_dir() itself.
• In sequential.rs, always construct the exact FrameworkCoreCache, including for params.clean builds.
• If params.clean_all is true, call remove() before any hydrate.
• Then hydrate as normal and store after a successful core/variant compile.
• A normal params.clean == true && params.clean_all == false must hydrate, allowing the compile stage to observe current .o, .d, and refreshed .cmdhash artifacts.

This brings the shared pipeline in line with the ESP32 behavior merged in #1093.

3. Add the CLI and daemon request

CLI touchpoints:

• crates/fbuild-cli/src/cli/args.rs: add a required value-enum scope (sketch or all) plus project/env/profile arguments.
• crates/fbuild-cli/src/cli/dispatch.rs: dispatch to a dedicated handler.
• crates/fbuild-cli/src/cli/clean.rs: resolve caller metadata and send the clean request.
• crates/fbuild-cli/src/cli/mod.rs: register the module.
• crates/fbuild-cli/src/daemon_client/types.rs and daemon_client.rs: add CleanRequest and POST /api/clean.

Recommended request shape:

{
  "project_dir": ".",
  "environment": "esp32dev",
  "scope": "all",
  "profile": "release",
  "caller_pid": 1234,
  "caller_cwd": "...",
  "pio_env": {}
}

Forward the PLATFORMIO_* snapshot because overlays participate in cache identity. Reuse OperationResponse unless a new response type provides a tested consumer-visible benefit.

Daemon touchpoints:

• crates/fbuild-daemon/src/models.rs: deserialize and test CleanRequest with a closed sketch|all enum/string contract.
• crates/fbuild-daemon/src/handlers/operations/clean.rs: implement the operation.
• crates/fbuild-daemon/src/handlers/operations/mod.rs: export it.
• crates/fbuild-daemon/src/main.rs: register POST /api/clean.

The handler must:

1. Resolve the client-relative project path, platformio.ini, environment, platform, profile, and build directory with the same helpers/precedence as the build handler (resolve_client_project_dir / resolve_build_dir and BuildLayout).
2. Acquire DaemonContext::project_lock(&project_dir) before deleting anything. A local CLI-only delete can race an active build and is not acceptable.
3. Run potentially large recursive deletion outside the async executor (use spawn_blocking, matching the daemon's other filesystem-heavy operations).
4. Return a useful success message naming scope, environment, and profile.

4. Reuse exact cache identity; never broaden deletion

The hard requirement is that standalone clean all deletes the same exact entries that build --clean-all would delete for the same project/env/profile/overlays:

• core: FrameworkCoreCache::new(...).path()
• ESP32 built-ins: FrameworkLibraryCache::new(...).path()

Extract a cache-identity/preparation helper from the existing orchestrator code and call it from both build and clean paths. It may parse configuration, inspect installed framework/toolchain metadata, scan source/header inputs, and calculate flags; it must stop before compiler/archive/linker subprocesses and before hydrate/store. If library include discovery is currently inseparable from compilation, split resolution/include discovery from compilation rather than approximating the key.

Do not:

• delete all of cache/core or cache/framework-libs;
• derive a second, simpler key for the clean command;
• omit profile, toolchain/framework identity, flags, overlays, or source/header content;
• encode the absolute project/build directory into a cross-project key;
• hydrate after eviction during a clean-only operation.

For the shared sequential path, expose a small engine helper that receives the already-resolved compiler, core/variant source list, overlay, and BuildParams, constructs the same FrameworkCoreCache, and either evicts (clean-only/all) or hydrates/stores (build). For ESP32, do the same for both the core cache and FrameworkLibraryCache. Keep this in existing engine/platform modules; do not add a crate.

Delete the project build directory last in a standalone clean operation. Cache identity preparation is allowed to create temporary/build-layout directories; deleting last guarantees the observable postcondition that the selected project output directory is absent.

5. Tests required

Unit/parser tests:

• clean sketch, clean all, explicit project, -e, --quick, and --release parse correctly.
• missing/invalid scope fails; quick+release conflicts.
• request JSON round-trips and rejects an unknown scope.
• FrameworkCoreCache::remove deletes its exact entry and leaves a sibling key untouched.
• ESP32 framework-library removal leaves a sibling key/profile untouched.

Behavior tests with real temporary directories:

• clean sketch removes the selected env/profile build directory but preserves another env/profile and both global cache sentinels.
• clean all removes the selected build directory plus the exact core/framework-library entries, while preserving mismatched env/profile/flags/content sibling entries.
• running either command twice succeeds.
• the clean operation takes the same project lock as build (a focused concurrency test may hold the lock and assert clean waits).
• shared sequential --clean hydrates a valid core cache; a counting/fake compiler proves cached core/variant TUs are not recompiled.
• shared sequential --clean-all evicts before hydrate, recompiles on the miss, and repopulates only the matching entry.
• clean-only tests assert no compiler, archiver, linker, or deploy subprocess is invoked.

Do not add a bare #[ignore]; any ignored integration test must include a concrete reason per repository policy.

Acceptance criteria

• fbuild clean sketch [PROJECT_DIR] [-e ENV] [--quick|--release] exists and removes only the selected project output directory.
• fbuild clean all ... additionally removes only the exact reusable framework entries applicable to that environment/profile.
• Both commands are daemon-coordinated, idempotent, and perform no build/deploy work.
• Shared-sequential orchestrators hydrate valid core caches during ordinary --clean builds.
• Shared-sequential --clean-all evicts the exact core entry before rebuilding.
• ESP32 behavior and cache-hit performance delivered by feat(build): preserve framework caches on clean #1093 remain intact.
• Tests prove selected-vs-sibling deletion, hydrate/store/evict ordering, and no-subprocess clean behavior.
• CLI reference documentation is updated (agents/docs/commands-reference.md and docs/reference/cli.md).

Focused validation

Use the repository wrappers only:

soldr cargo test -p fbuild-cli cli::tests
soldr cargo test -p fbuild-daemon models::tests
soldr cargo test -p fbuild-build-engine framework_core_cache
soldr cargo test -p fbuild-build-esp framework_library_cache --lib
soldr cargo check -p fbuild-cli -p fbuild-daemon -p fbuild-build-engine -p fbuild-build-esp --all-targets
soldr cargo clippy -p fbuild-cli -p fbuild-daemon -p fbuild-build-engine -p fbuild-build-esp --all-targets -- -D warnings
bash ./test

Related work

• feat(build): preserve framework caches on clean #1093 - merged implementation of ESP32 cache retention/eviction and build/deploy flags.
• agents/docs/path-conventions.md - authoritative cache/build roots and cross-project key rules.