Repository navigation
perf(packages): overlap download/extract/install in cold pioarduino-resolve (~185-237s serialized) #953
Closed as not planned
Description
Context
#942 Phase-1 baseline, cold scenario: pioarduino-resolve costs 185–237 s of the 700–808 s cold build. Harness stderr shows repeated sequential esp32-xtensa-gcc ... downloading lines followed by a single extracting — the pipeline downloads, then extracts, then installs, largely serially. fbuild-packages has a download_all() that spawns per-URL tasks, but the ESP32 toolchain/platform/framework ensure path doesn't fan out, and extraction doesn't overlap with the next download.
Additionally fw-libs (162 s cold) only starts after resolve completes — compile of framework libs could begin as soon as its toolchain + sources are present rather than waiting for all packages.
Proposal
- Pipeline the package ensure path: concurrent downloads (bounded), extract-while-downloading (stream or start extract per completed archive), install-while-extracting.
- Start dependent build phases as soon as their specific inputs are ready instead of barriering on full resolve (e.g. fw-libs can start when toolchain + core sources exist, while libraries still download).
Acceptance criteria
- Cold
pioarduino-resolvewall time (or its serialized portion) drops by ≥ 40% in the meta: profiling-driven build performance burndown (Docker Linux, NightDriverStrip, cold vs hot cache) #942 harness on the same network. - Off-CPU profile shows downloads overlapping extraction/install instead of sequential stair-steps.
- Cold total drops correspondingly (~774 s baseline).
- Before/after harness numbers in the PR description.
Part of #942.
Activity
perf(esp32): download GCC toolchain concurrently with framework + SDK…
fix(esp32): surface both errors if toolchain AND framework resolve fa…
zackees commentedon Jul 6, 2026
In progress on branch
fix/953-cold-overlap(local worktree, not pushed): clippy + tests green, cold-build A/B profiling incomplete. Full handoff + remaining overlap candidates in #990.zackees commentedon Jul 7, 2026
Measured the #953 branch against the #942 Docker cold profile harness on the same Windows/Docker Desktop host and same network, two cold runs per side.
Baseline:
origin/mainat7f28fc67(after #989)python ci/docker-profile/run_profile.py -n 2 --scenarios cold --out ci/docker-profile/out/codex-baseline-main-7f28fc67-py.claude/worktrees/953-baseline/ci/docker-profile/out/codex-baseline-main-7f28fc67-pyCandidate:
fix/953-cold-overlapate31bcc59(rebased onto currentorigin/mainincluding #991)python ci/docker-profile/run_profile.py -n 2 --scenarios cold --skip-image --out ci/docker-profile/out/codex-candidate-953-e31bcc59.claude/worktrees/953/ci/docker-profile/out/codex-candidate-953-e31bcc59Focused local checks on the rebased branch also pass:
soldr cargo fmt --allsoldr --no-cache cargo test -p fbuild-packages esp32_framework --message-format=shortsoldr --no-cache cargo check -p fbuild-build -p fbuild-packages --message-format=shortgit diff --check origin/main..HEADConclusion: this does not meet the >=40% cold
pioarduino-resolveacceptance bar and the cold total is not improved (median regressed from 855.1s to 903.5s). Closing as diminishing-return rather than shipping a concurrency change that does not have measurable benefit.