You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
audit: replace mock-based tests with integration tests where the contract is a real integration boundary (#826 followup) #838
The #826 gotcha-sweep audit identified several test files that lean heavily on mocks to stand in for what is actually an integration boundary (subprocess invocation, filesystem fixtures, real serial-port behavior). Mocking those surfaces makes the tests pass in isolation but lets behavioral drift slip through — the mock and the real implementation can disagree, and the test won't notice.
crates/fbuild-build/tests/compile_many_two_stage.rs — uses a mock compile dispatcher instead of a real toolchain
crates/fbuild-daemon/tests/test_emu_exit_code.rs — fakes the emulator runner instead of exercising the real qemu/simavr path
Plus the broader "mocking-as-default" pattern in compile_many.rs, compile_many_stage2_perf.rs, and the unit tests under fbuild-deploy/src/esp32/tests.rs
(Full list to be enumerated by a fresh audit script as part of this issue's scope.)
Why this is NOT a dylint
Dylint operates at the rustc AST level. It cannot distinguish a "mock" from a "test-only struct" — both are just structs in #[cfg(test)] code. The semantic decision ("this surface should be exercised against a real implementation") is fundamentally a code-review / design call.
Recommendation
Add a CodeRabbit rule (.coderabbit.yaml) that flags new mock-typed traits / structs in test files for human review with the prompt "Is the mocked boundary a real integration surface (subprocess, FS, serial)? If yes, prefer tempfile + real binary."
Run a one-time audit of the existing offenders listed above, decide per-file whether to keep mock-based (acceptable for unit tests of pure logic) or migrate to a real fixture (tempfile::TempDir, real Command, etc.)
Document the policy in CLAUDE.md §"Development Philosophy: TDD" — "test real behavior; use mocks only when the abstraction has no concrete dependency you can stand up cheaply".
Activity
zackees commentedon Jun 30, 2026
Rolled up under #867 (audit dashboard meta).