Meta: fbuild bloat — first-class, end-to-end bloat analysis with build_info-driven toolchain resolution (no manual nm/c++filt, no PlatformIO, no side scripts) #434

TL;DR

fbuild symbols (#424 + #427) works, but every real-world invocation
in the recent FastLED audit (PR #2871 trail) had to be propped up
with side tools — explicit --nm, PlatformIO builds, custom Python
aggregators, manual cp of dev binaries, hand-typed
.pio/build/<env>/firmware.elf paths. This meta issue is the
migration plan to a single end-to-end command that does the whole
thing using only fbuild + build_info.json.

Ideal user surface (proposed)

$ fbuild bloat .            # or: fbuild bloat path/to/sketch
Building esp32s3 (release)...
Analyzing 4,591 symbols (nm via /home/.../xtensa-esp32s3-elf-nm)...
Demangling via /home/.../xtensa-esp32s3-elf-c++filt...

Bloat report:
  Flash: 388,380 B across 3,685 symbols
  RAM:    37,460 B across   906 symbols

Written to:
  /abs/path/.fbuild/build/esp32s3/bloat-report/report.json
  /abs/path/.fbuild/build/esp32s3/bloat-report/report.md

No flags. No manual paths. No --nm. Toolchain paths come from the
build_info.json that the link step just emitted in
.fbuild/build/<env>/.

Why bloat and not symbolize / symbols

• symbolize in industry usage means address→symbol resolution for
  crash dumps and stack traces (e.g. llvm-symbolizer). Wrong word.
• cargo bloat and Google's bloaty are the de-facto names for
  per-symbol size analysis. Aligning with that vocabulary lowers
  the learning curve.
• Existing fbuild symbols (shipped in feat(symbols): standalone fbuild symbols <elf> fine-grained bloat report #424) can stay as a hidden
  alias for one release, then deprecate.

Proposed: fbuild bloat <path> (with fbuild symbols kept as
hidden alias for back-compat through 2.3.x).

Side-tool inventory — every instance from the recent audit

Each row maps to a concrete change required in this meta. Sourced
verbatim from the FastLED #2773 / PR #2871 trail.

1. PlatformIO used as the build backend

Why this matters: PIO doesn't emit build_info.json in
fbuild's shape, so the symbolizer can't auto-resolve toolchain
paths. Forces every invocation to pass --nm by hand. The user's
constraint: "please only use the build_info.json that's generated
that should have the abs tool paths".

2. Manual cross-toolchain invocations

xtensa-esp32s3-elf-size -A firmware.elf       # section sizes
xtensa-esp32s3-elf-nm --print-size --size-sort --reverse-sort -C firmware.elf   # top symbols

Should be: subsumed by fbuild bloat. Section sizes already in
JSON (sections[]); top symbols in symbols[]; both rendered in
the MD report.

3. Hand-written Python aggregators

The FastLED side carried three custom scripts in
.claude/symbolaudit/ for the duration of the audit:

4. Explicit --nm paths on every single invocation

Every command in the audit looked like:

uv run fbuild symbols .build/pio/esp32s3/.pio/build/esp32s3/firmware.elf \
    --nm ~/.platformio/packages/toolchain-xtensa-esp-elf/bin/xtensa-esp32s3-elf-nm.exe \
    --json out.json

The --nm flag was passed 7+ times. Should be zero.

Blocker: #428 (extend BuildInfo with
nm_path / cppfilt_path / readelf_path / objdump_path).

5. Manual cp of dev binary into .venv/Scripts/fbuild.exe

cp /c/Users/.../target/release/fbuild.exe .venv/Scripts/fbuild.exe

Every time the agent iterated on fbuild Rust source. Pre-existing
friction; tracked by #429 (cross-project hook propagation) and
FastLED #2871 (soldr enforcement at fastled10's root).

6. Manual ELF/map path typing

Every invocation hardcoded
.build/pio/esp32s3/.pio/build/esp32s3/firmware.elf and
firmware.map. The new project-mode resolver (fbuild symbols .)
exists post-#424 but was still rarely used because it only finds
PIO output trees today — fbuild's own .fbuild/build/<env>/ is
also discovered, but the user never built with fbuild during the
audit (see item 1).

Migration plan (phased)

Phase 1 — BuildInfo toolchain paths (blocker for everything below)

Tracked at #428. Adds nm_path / cppfilt_path / readelf_path / objdump_path to BuildInfo plus an aliases block for PIO
consumer compatibility. Each platform orchestrator (AVR, ESP32,
RP2040, STM32, Teensy, ESP8266, NRF52, SAM, Renesas, Apollo3, LPC,
Silabs, CH32V) populates the four new fields.

Acceptance: after fbuild build ., the emitted
build_info_<env>.json carries non-empty absolute paths for all
four tools on every native-orchestrator platform.

Phase 2 — fbuild bloat (rename + remove flags)

• Rename Commands::Symbols → Commands::Bloat in
  crates/fbuild-cli/src/cli/args.rs. Keep Symbols as a hidden
  alias deprecated in 2.4.0.
• Drop --nm / --cppfilt / --map from the public surface (still
  accepted as overrides, but not shown in --help).
• Make the input arg purely a project path. ELF resolution stays
  via discover_elf_in_project.
• Look up tools via build_info.json::nm_path /
  cppfilt_path. Error out with a clear message ("run fbuild build first") when build_info.json is missing the new fields.

Acceptance: fbuild bloat . works zero-flag against a
fbuild-built project on every supported platform.

Phase 3 — Default output dir under build tree + path printing on exit

• When neither --json nor --output-dir is given, default to
  .fbuild/build/<env>/bloat-report/. Always write both
  report.json and report.md.
• Print absolute paths on exit (see TL;DR example).
• fbuild build . should optionally chain into bloat analysis via
  --bloat flag (the existing --symbol-analysis flag morphs into
  --bloat, with the legacy spelling kept as alias).

Acceptance:

fbuild bloat . 2>&1 | grep -E "report\.(json|md)"
# emits exactly two lines, both absolute paths under .fbuild/build/<env>/bloat-report/

Phase 4 — fbuild bloat-diff <a> <b>

Subsumes the ad-hoc Python diff.py. Inputs:

• Two paths (each may be a project dir, a report.json, or an
  ELF — same resolution as fbuild bloat).
• Optional --top N (default 25).
• Optional --region flash|ram (default flash).

Output: same dual report.json + report.md shape, but content
is a delta table (added / removed / grew / shrunk per symbol;
per-archive rollup of net change).

Acceptance: running fbuild bloat-diff <3.10.3-project> <master-project> reproduces the #2773 audit report from the
recent FastLED audit verbatim (modulo timestamps).

Phase 5 — Default-on --bloat after fbuild build (opt-out)

Same way fbuild build emits build_info.json by default today,
make it emit bloat-report/{report.json,report.md} by default
post-link. Opt-out via --no-bloat-report if the build needs to
stay narrowly scoped (CI matrix where bloat reports would blow up
the artifact size).

Acceptance: fbuild build . on an esp32s3 Blink project drops
bloat-report/report.json + report.md alongside the ELF without
any extra flag.

Integration test — ESP32-S3 + FastLED master Blink

Lives in crates/fbuild-cli/tests/bloat_esp32s3_fastled_master.rs
or similar. Behaviour (Rust pseudocode):

#[test]
#[ignore = "network: fetches FastLED master + downloads esp-idf toolchain"]
fn bloat_esp32s3_blink_fastled_master_end_to_end() {
    // 1. Materialise a fbuild project skeleton in a temp dir:
    //    - platformio.ini selecting board=esp32s3
    //    - src/main.cpp that #include <FastLED.h> + a NEOPIXEL blink loop
    //    - lib_deps = "https://github.com/FastLED/FastLED#master"
    let project = TempProject::scaffold_esp32s3_blink_fastled_master();

    // 2. fbuild build .   (uses fbuild's native esp32 orchestrator; no PIO)
    let build_result = run_fbuild(&["build", project.path()])
        .expect("fbuild build succeeded");

    // 3. Verify build_info.json has the four toolchain paths.
    let build_info = project.read_build_info("esp32s3");
    assert!(!build_info["nm_path"].as_str().unwrap().is_empty());
    assert!(!build_info["cppfilt_path"].as_str().unwrap().is_empty());
    assert!(!build_info["readelf_path"].as_str().unwrap().is_empty());
    assert!(!build_info["objdump_path"].as_str().unwrap().is_empty());

    // 4. fbuild bloat .  (no flags, no --nm)
    let bloat_stdout = run_fbuild(&["bloat", project.path()])
        .expect("fbuild bloat succeeded");

    // 5. Report files exist at the documented path.
    let report_dir = project.path().join(".fbuild/build/esp32s3/bloat-report");
    assert!(report_dir.join("report.json").is_file());
    assert!(report_dir.join("report.md").is_file());

    // 6. stdout prints both absolute paths on exit.
    assert!(bloat_stdout.contains("report.json"));
    assert!(bloat_stdout.contains("report.md"));

    // 7. Content invariants — values from the manual #2773 audit. Bounds
    //    are generous so FastLED master drift doesn't flake the test.
    let report: serde_json::Value =
        serde_json::from_str(&std::fs::read_to_string(report_dir.join("report.json")).unwrap()).unwrap();
    let total_flash = report["total_flash"].as_u64().unwrap();
    let total_ram = report["total_ram"].as_u64().unwrap();
    let symbols = report["symbols"].as_array().unwrap();
    let sections = report["sections"].as_array().unwrap();

    assert!(
        (300_000..500_000).contains(&total_flash),
        "expected ~388 KB flash, got {}", total_flash
    );
    assert!(
        (25_000..60_000).contains(&total_ram),
        "expected ~37 KB ram, got {}", total_ram
    );
    assert!(symbols.len() > 2_000, "expected >2000 sized symbols");
    assert!(sections.len() > 100, "expected per-archive sections");

    // 8. Map-derived synthesis covered (PR #427 invariant): the FastLED
    //    chipset constructor's rodata pool is named, not bucketed at
    //    object level.
    let synth_count = symbols.iter()
        .filter(|s| s["source"].as_str() == Some("map-derived"))
        .count();
    assert!(synth_count > 100, "expected >100 map-derived synthetic owners");

    // 9. Markdown report renders a top-flash table.
    let md = std::fs::read_to_string(report_dir.join("report.md")).unwrap();
    assert!(md.contains("# Symbol analysis"));
    assert!(md.contains("## Top"));
    assert!(md.contains("flash symbols"));
    assert!(md.contains("| Bytes |"));
    assert!(md.contains("fl::"));  // a FastLED symbol made it into top-25
}

Why pin FastLED master: the integration test exercises the
realistic case (latest FastLED, biggest binary, most complex symbol
graph). The >2000 symbols / >100 map-derived invariants are
generous so the test doesn't go red the moment FastLED merges a
size-affecting refactor — they pin the shape of the report,
not the exact bytes.

Suggested infra:

• #[ignore] by default so the test isn't network-dependent for
  ordinary cargo test runs.
• Run in nightly CI matrix (workflow_dispatch + scheduled
  weekly) so FastLED master regressions surface here too.
• Cache the esp-idf toolchain via the existing setup-soldr
  zccache flow — second run should be < 30 s.

Friction inventory (this issue resolves all of them)

Sub-issues to file when this lands

• fbuild symbols: ingest toolchain paths from build_info.json + extend schema with nm/c++filt/readelf/objdump #428 (already filed) — Phase 1 build_info toolchain paths
• Phase 2 — rename symbols → bloat, drop flag dependencies
• Phase 3 — default-output-dir + exit-message paths
• Phase 4 — fbuild bloat-diff
• Phase 5 — auto-bloat-after-build
• Integration test PR (Phase 2 + Phase 3 prerequisite)

Definition of done for this meta

This meta closes when all five phases land and the integration
test above is green on the nightly esp32s3 + FastLED-master matrix
for two consecutive weeks (catches transient toolchain / cache
flakes).

Related

• PR feat(symbols): standalone fbuild symbols <elf> fine-grained bloat report #424 (initial fbuild symbols subcommand) — MERGED
• PR feat(symbols): map-derived synthesis for anonymous rodata pools (closes #425) #427 (map-derived synthesis closing fbuild symbols: extract owning symbol from .rodata.<mangled>.str1.N input sections (377 KB of rodata currently buckets at object level) #425) — MERGED
• Issue fbuild symbols: extract owning symbol from .rodata.<mangled>.str1.N input sections (377 KB of rodata currently buckets at object level) #425 — symbol-attribution gap (closed by feat(symbols): map-derived synthesis for anonymous rodata pools (closes #425) #427)
• Issue fbuild symbols: ingest toolchain paths from build_info.json + extend schema with nm/c++filt/readelf/objdump #428 — BuildInfo toolchain-path schema (Phase 1 blocker)
• Issue Hooks: tool_guard.py (force-soldr-for-cargo) doesn't fire when agent works in fbuild worktree from another project's session #429 — Claude Code hooks load at session root only
• FastLED #2871 — soldr enforcement at fastled10's PreToolUse hook
• FastLED #2773 — ESP32-S3 binary-size meta tracker (the
  motivating audit that surfaced every side-tool here)