id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay

Finding

PR #1387's new benchmark harness (pkgs/id/benches/world_bench.rs, head commit b6f5b720e9fc7d117f44b46181818d114dbad3aa) ships a guest_replay benchmark that never replays anything through the guest. It builds a journal of N Committed chat events, calls WorldCore::restore, and then pushes restored.replay through its CountingProgram — but on current main, restore does not put committed chat events into replay at all, so the guest pass iterates an empty vec and guest.seen stays 0. The reported guest_replay number is restore-only time, mislabeled as guest replay.

Evidence

PR: wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387 (OPEN, slice 1 of id(ci): F11.8 benchmark harness with committed baselines #539)
pkgs/id/benches/world_bench.rs:121 — bench_guest_replay builds the journal from chat_event(...) entries (WorldEventKind::Chat).
pkgs/id/benches/world_bench.rs:166 — for event in &restored.replay { guest.update(event)?; } — the loop that is supposed to be the measured guest workload.
Current main pkgs/id/src/world/core.rs:347 (verified at 37418e56): in WorldCore::restore, the JournalEntry::Committed match arm is WorldEventKind::System { .. } | WorldEventKind::Chat(_) => {} — chat events are applied to core state via commit_prepared but are not pushed to replay. Only Input committed events (core.rs:344) and journal System entries (core.rs:317) land in replay.
The bench's own inline comment half-notices this ("count the guest pass over the full journal's events as the replay workload when restore() consumed them into core state directly") but the code does not do that — it still iterates restored.replay.

Why it matters

Slice 2 of #539 commits baselines recorded from this harness, and slice 3's CI job diffs medians against them. A guest_replay baseline that contains zero guest work will (a) understate the real replay cost the benchmark exists to track, and (b) make any future fix to the bench look like a massive regression when the guest pass starts doing real work. The failure is silent: the bench prints a plausible-looking line either way.

Acceptance criteria

guest_replay actually pushes the N committed events through the guest (e.g. iterate the journal's committed events / core history, or build the journal from event kinds that restore returns in replay), OR the benchmark is renamed/re-scoped to state honestly that it measures restore only.
The bench asserts the guest saw the expected count (e.g. guest.seen == events, or restored.replay.len() is checked non-empty) so a future restore-semantics change cannot silently re-empty the workload.
cargo bench --bench world_bench -- --quick --only guest_replay (or the just/nix runner equivalent) completes with the assertion in place.

Refs #539. Found by pr-diff-review-b reviewing PR #1387's diff against current main.

CODE-494

🤖 Agent note · filed

Agent: pr-diff-review-b
Session / ID: pr-diff-review-b cron run 2026-10-27 06:18 CDT (lane B first run; B-state initialized this run)
Found via: filed this run
Task: PR wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387's guest_replay benchmark (pkgs/id/benches/world_bench.rs:121,166) pushes restored.replay through its counting guest, but current main WorldCore::restore (pkgs/id/src/world/core.rs:347, verified at 37418e5) never puts committed Chat events in replay — only Input committed events (:344) and journal System entries (:317) land there. The bench's journal is all chat events, so the guest sees zero events and the reported number is restore-only. Enqueued as a triage packet to repo-health-code-pilot.
Expectation: triage confirms against PR wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387 / current main; done when the bench measures a real guest pass (or is honestly re-scoped to restore-only) and asserts the guest saw the expected event count, per the issue's acceptance criteria.
Links: PR wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387 · issue id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403 · parent id(ci): F11.8 benchmark harness with committed baselines #539

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), lane D
Session / ID: queue task 1391 (triage)
Found via: queue claim
Task: Triage id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403 — verify the guest_replay empty-replay finding against PR wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387 and current main, then route the fix
Expectation: verdict posted on id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403 with fresh file:line evidence; follow-up fix routed as a named queue task; labels truthful
Links: id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403 · wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387 · id(ci): F11.8 benchmark harness with committed baselines #539

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), Lane C shift
Session / ID: queue task 1742
Found via: queue claim
Task: Fix the guest_replay benchmark (id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403): build the replay workload from events restore() legitimately replays (or honestly re-scope it), assert the guest-seen count so an empty replay fails loudly, and correct the guest_replay line in baseline.txt + BASELINES.md. WorldCore::restore semantics stay untouched.
Expectation: effort m; done = fix PR opened with CI green (bench assertion in place), or parked with a precise resume condition.
Links: issue id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403 · source PR wip(id): F11.8 slice 1 benchmark harness — four world benchmarks + just/nix runners (#539) #1387 (merged)

🤖 Agent note · parked

Agent: queue worker (repo-worker-code), Lane C shift
Session / ID: queue task 1742
Found via: queue claim
Task: Parked awaiting CI + Codex on PR fix(id): guest_replay bench replays committed input events and asserts the guest-seen count (#1403) #2885 (head 054bbfbd4). Fix complete and locally verified on the reference laptop: assertion fires on the old workload (guest replay saw 0 of 200 committed events), quick run passes, full-size baseline regenerated (median=0.170, two runs agreed). Checks had not registered ~4 min after the PR opened; Codex review requested.
Expectation: resume condition — read PR fix(id): guest_replay bench replays committed input events and asserts the guest-seen count (#1403) #2885 checks + Codex findings fresh; merge when green with findings dispositioned, then close this issue.
Links: PR fix(id): guest_replay bench replays committed input events and asserts the guest-seen count (#1403) #2885 · branch queue/1742-1403-guest-replay · worktree ~/git/fleet/1742-1403-guest-replay (laptop)

fix(id): guest_replay bench replays committed input events and assert…

🤖 Agent note · done

Agent: queue worker (repo-worker-code), lane C shift
Session / ID: queue task 1742
Found via: queue claim — resumed PR fix(id): guest_replay bench replays committed input events and asserts the guest-seen count (#1403) #2885 from checkpoint
State: PR fix(id): guest_replay bench replays committed input events and asserts the guest-seen count (#1403) #2885 @054bbfbd4 verified fresh: Codex review completed clean (no findings, zero review threads); mergeState CLEAN. CI note: no checks ever registered because no PR-triggered workflow covers this change — id-check.yml has no pull_request trigger (push-to-main/schedule/dispatch only) and id-bench.yml is main-tier only by design (ci: Phase 2 — PR-tier slimming (basic validation on PRs, full suite on main) #1781). Verification therefore rests on the prior lane's local run (assertion fires on the old workload; full-size baseline regenerated, median=0.170) plus Codex. The post-merge push to main will run id-check + id-bench on the merged tree.
Outcome: PR fix(id): guest_replay bench replays committed input events and asserts the guest-seen count (#1403) #2885 squash-merged as f2e9575; this issue auto-closed.

Navigation Menu

Repository navigation

id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403

Finding

Evidence

Why it matters

Acceptance criteria

39 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

id(bench): PR #1387 guest_replay benchmark replays zero events — restore() never puts committed chat events in replay #1403

Description

Finding

Evidence

Why it matters

Acceptance criteria

Activity

developing-today commented on Oct 27, 2026

dezren39 commented on Oct 27, 2026

dezren39 commented on Oct 28, 2026

39 remaining items

dezren39 commented on Oct 31, 2026

dezren39 commented on Oct 31, 2026

dezren39 commented on Oct 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions