You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bench: intermittent per-command daemon restart ('daemon binary updated') adds ~200 ms to every fbuild command #1476
Some benchmark-build-comparison.yml runs show fbuild warm ≈ 251 ms (normally ≈ 46 ms) and cold inflated by about 200 ms. In those runs every fbuild command in benchmark.log prints daemon binary updated, restarting..., and the value holds steady across all 3 trials. It comes from the direct-HTTP acquisition path in crates/fbuild-cli/src/daemon_client.rs (ensure_direct_daemon_running → should_restart_daemon: same version, CLI-side sibling fbuild-daemon mtime > the daemon's reported source_mtime).
It is pre-existing and intermittent
main nightly history has one run at warm 249.8 ms among 73 runs. That was before the zccache repin.
four runs of the same or similar code (36235583298, 36236087937, 36236206722, 36236358015);
a forced cache-miss run (36239211262).
A bad run is bad for its entire duration, which points to a per-job condition rather than per-command noise.
Ruled out
An mtime change during the run: stat of target/release/fbuild-daemon (mtime and inode) stayed constant across a run. That was checked in the debug steps, which happened to be good runs.
Inherited stdio holding Command::output() open: the daemon's fds 0/1/2 are /dev/null, /dev/null and daemon.log.
soldr cache hit vs miss for fbuild-daemon: the forced-miss run was good.
Catch a bad run with RUST_LOG=fbuild_cli=info on the benchmark step. The daemon needs restart (daemon v… mtime=…, cli v… mtime=…) tracing line gives both sides of the comparison.
Independently, should_restart_daemon could compare a build identity (the git sha or a build id the daemon reports) instead of raw mtimes. That would make the same-version dev-rebuild check immune to anything that perturbs mtimes.
The runner passes FBUILD_PERF_LOG / FBUILD_PERF_LOG_JSON only to the timed fbuild build commands, not to the untimed clean cache that (re)starts the daemon. So the phase log is written only when a timed build command itself starts the daemon, which is exactly the restart mode. In normal runs cold_phases_ms is empty.
That makes two things to fix together:
The benchmark runner: pass the perf-log env to the cold-prep clean cache / daemon-restart commands too, or have the CLI forward the perf-log request to an already-running daemon.
The restart mode itself: see above. One of its visible effects is that the phase log gets populated.
Activity
zackees commentedon Sep 26, 2026
Additional finding: across 5 runs, per-phase data (
cold_phases_ms, #1465) is present only in the restart runs:The runner passes
FBUILD_PERF_LOG/FBUILD_PERF_LOG_JSONonly to the timedfbuild buildcommands, not to the untimedclean cachethat (re)starts the daemon. So the phase log is written only when a timed build command itself starts the daemon, which is exactly the restart mode. In normal runscold_phases_msis empty.That makes two things to fix together:
clean cache/ daemon-restart commands too, or have the CLI forward the perf-log request to an already-running daemon.feat(daemon): diagnose the per-command daemon restart mode (#1476) (#…