id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally

id(world): `admin_history_bytes` re-serializes the entire retained admin history on every accepted event — track retained bytes incrementally

Scope: PR #3998 head 47937fe3e73707eaefbaddc6d196a45f26ddc366 (fix(id): export host events through authorized journal history). The admin-history machinery does not exist on main yet — it is introduced by that PR — so this is a follow-up on the PR's new code, cross-linked both ways. Independent origin: Codex P2 review comment r4437438581 on PR #3998; re-verified line-by-line against the PR head this run, and left unfiled by the 2026-11-02 pr-comment-review run only because that run hit its per-repo filing cap.

Evidence (all line numbers on PR head `47937fe3`)

pkgs/id/src/world/core.rs:7-12 — the helper itself:

fn admin_history_bytes<'a>(events: impl IntoIterator<Item = &'a WorldEvent>) -> usize {
    events
        .into_iter()
        .map(|event| serde_json::to_vec(event).map_or(usize::MAX, |bytes| bytes.len()))
        .fold(0usize, usize::saturating_add)
}

Every call allocates and JSON-serializes every event it is given, just to sum the byte lengths.

pkgs/id/src/world/core.rs:1317-1336 — record_admin_history pushes the new event, then evaluates the helper in the trim-loop condition:
```
while self.admin_history.len() > self.limits.retained_events.min(MAX_ADMIN_HISTORY_EVENTS)
    || admin_history_bytes(&self.admin_history) > MAX_ADMIN_HISTORY_BYTES
```
In steady state (length within cap) the byte total is computed on every call, and each pop_front re-evaluates the condition — so trimming k events costs k+1 full re-serializations of the remaining deque.
Call frequency: record_admin_history runs on the accept path — pkgs/id/src/world/core.rs:1312 (inside event acceptance), also core.rs:457, core.rs:650, and pkgs/id/src/world/actor.rs:445. Every accepted input, chat message, or host event pays the full scan even when no admin export is ever requested.
Bounds: pkgs/id/src/world/mod.rs:57 MAX_ADMIN_HISTORY_EVENTS = 4096; mod.rs:59 MAX_ADMIN_HISTORY_BYTES = 64 MiB. With the default 1,024-event retention and valid 16 KiB inputs, one accepted event can re-encode ~16 MiB of JSON; at the 4,096-event cap, up to the full 64 MiB byte budget is re-encoded per event.
Secondary edge in the same helper: a single event whose serialization fails counts as usize::MAX, so the total saturates and the trim loop would drain the entire retained history one pop (and one full re-scan) at a time.
WorldCore is deliberately serialized: core.rs:28 — "Keep it behind one actor/mutex so event sequencing is serialized." The re-serialization is synchronous work inside that single world actor, so it delays subsequent inputs, ticks, and view requests for everyone in the world.
One legitimate one-shot use exists: restore_admin_history (core.rs:1349) validates a checkpoint's history once at restore time. That call can stay, or reuse the same cached sizes.

Why it matters

This is an O(retained-history) tax on the hottest path in the world actor, paid per accepted event, forever, in exchange for a byte total that changes by exactly one event's size per call. Busy worlds (the ones that accumulate a large retained history) get slower per event the more history they retain — precisely backwards. The fix is mechanical and local: cache each event's encoded size when it enters the deque and maintain a running total as events enter and leave.

Acceptance criteria

record_admin_history is O(1) in the retained-history size per accepted event (amortized): encoded sizes are cached per retained event and a running byte total is maintained incrementally on push/pop; no per-event full re-serialization on the accept path.
Trim behavior is unchanged: the same events are retained/evicted under both the event-count cap (retained_events.min(MAX_ADMIN_HISTORY_EVENTS)) and the byte cap (MAX_ADMIN_HISTORY_BYTES), and admin_history_complete flips exactly as before.
restore_admin_history's byte-bound validation still rejects an over-budget checkpoint history (it may reuse the cached sizes or keep the one-shot scan — restore is not the hot path).
The serialization-error edge is handled deliberately (an event that cannot be serialized cannot silently poison the running total or drain the whole history); the chosen behavior is pinned by a test.
A regression test or benchmark demonstrates the improvement (e.g. a test with a large retained history asserting the trim outcome, plus a comment/measurement showing per-event cost no longer scales with history size), and the existing world test suite passes.

Cross-links: PR #3998 (introduces the code), Codex review comment r4437438581.

🤖 Agent note · filed

Agent: new-issue-creator job (lane A, new-issue-creator-code-a)
Session / ID: cron job new-issue-creator-code-a, run 2026-11-02 16:35 CST (2026-11-02T22:35Z)
Found via: filed this run
Task: Filed this issue: admin_history_bytes (PR fix(id): export host events through authorized journal history #3998 head 47937fe3) re-serializes the entire retained admin history on every accepted event; track retained bytes incrementally instead. Origin: Codex P2 r4437438581 on PR fix(id): export host events through authorized journal history #3998, left unfiled by pr-comment-review's per-repo cap; re-verified line-by-line on the PR head this run (helper at pkgs/id/src/world/core.rs:7-12, trim loop :1328-1329, accept-path callers core.rs:1312 / actor.rs:445, caps mod.rs:57,59). Cross-linked on PR fix(id): export host events through authorized journal history #3998.
Expectation: effort/s — triage confirms on the PR head (or on main once PR fix(id): export host events through authorized journal history #3998 lands) and routes the incremental byte-tracking fix.
Links: PR fix(id): export host events through authorized journal history #3998 · queue packet to follow

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), OpenChamber scheduled lane
Session / ID: scheduled run · task 6551 · lane 7b47536b-c162-444c-be8a-3a9538cc4cb5
Found via: atomic claim on repo-health-code-pilot
Task: verify the admin_history_bytes hot-path finding at current PR fix(id): export host events through authorized journal history #3998 head and route the narrow fix; no changes to Drew-owned PR fix(id): export host events through authorized journal history #3998.
Expectation: source evidence at current head plus a distinct, scoped follow-up packet if confirmed.
Links: PR fix(id): export host events through authorized journal history #3998 · issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633

🤖 Agent note · parked

Agent: queue worker (repo-worker-code), lane code-b
Session / ID: task 6562
Found via: queue claim
Task: task 6562 / issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633 — fix re-serializing retained admin history per event, gated on PR fix(id): export host events through authorized journal history #3998 landing on main
Expectation: picked up and re-validated fresh at 2026-11-03T02:21Z: PR fix(id): export host events through authorized journal history #3998 still OPEN, author dezren39, unmerged (mergedAt null), head 47937fe; issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633 still OPEN, no held/user-owned labels. Gate unmet — packet forbids touching PR fix(id): export host events through authorized journal history #3998 or creating a competing change, so task 6562 is parked again (30m). Resume only after PR fix(id): export host events through authorized journal history #3998 merges to main.
Links: PR fix(id): export host events through authorized journal history #3998 (gate), issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), lane code-j
Session / ID: task 6562
Found via: queue claim
Task: Gate revalidated fresh: PR fix(id): export host events through authorized journal history #3998 is MERGED (2026-11-03T02:48:07Z, merge commit 07b7d70), so the admin-history code is on main. Implementing incremental retained admin-history byte tracking for issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633: cached per-event encoded sizes + incrementally maintained total, preserving trim/checkpoint semantics.
Expectation: effort/s — done = PR opened and merged with world tests passing, issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633 closed.
Links: issue id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633, PR fix(id): export host events through authorized journal history #3998 (merged, gate)

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), lane B shift
Session / ID: task 6562
Found via: queue claim
Task: Gate revalidated fresh: PR fix(id): export host events through authorized journal history #3998 MERGED at 2026-11-03T02:48:07Z (merge commit 07b7d70, ancestor of origin/main), so the gated fix for id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633 is now unblocked. Resuming the existing laptop worktree ~/git/fleet/6562-4633-admin-history on branch queue/6562-4633-admin-history-bytes, which already holds an uncommitted incremental byte-ledger implementation in pkgs/id/src/world/core.rs; verifying it, adding regression coverage, and opening the PR.
Expectation: effort/s — done = PR open with the incremental admin_history_bytes accounting, trim/restore semantics preserved, targeted world tests passing.
Links: branch queue/6562-4633-admin-history-bytes; gated on PR fix(id): export host events through authorized journal history #3998 (now merged)

Merge pull request #4747 from developing-today/queue/6562-4633-admin-…

🤖 Agent note · done

Agent: queue worker (repo-worker-code), lane B shift
Session / ID: task 6562
Found via: queue claim
Task: Implemented the incremental admin-history byte accounting for id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633 now that PR fix(id): export host events through authorized journal history #3998 is merged to main. PR fix(id/world): track retained admin-history bytes incrementally (#4633) #4747 merged (merge commit 66807e6): AdminHistoryByteLedger caches each retained event's encoded size and maintains the total on push/pop, so record_admin_history no longer re-serializes the retained history per accepted event. Unserializable events count as usize::MAX (legacy rescan semantics, forces a trim); restore keeps its one-shot over-budget rejection and rebuilds the ledger.
Expectation: done — cargo test --manifest-path pkgs/id/Cargo.toml world on the laptop: 479 passed, 0 failed, including 4 new ledger regression tests (unserializable-as-MAX, count-trim sync, byte-cap trim, restore sync). Codex review on PR fix(id/world): track retained admin-history bytes incrementally (#4633) #4747 completed with no findings.
Links: PR fix(id/world): track retained admin-history bytes incrementally (#4633) #4747 (merged) · branch queue/6562-4633-admin-history-bytes · commit e9eae6c

🤖 Agent note · done (correction to the test evidence in the previous note)

Agent: queue worker (repo-worker-code), lane B shift
Session / ID: task 6562
The earlier note's "479 passed, 0 failed" figure for cargo test --manifest-path pkgs/id/Cargo.toml world was wrong: two concurrent runs of that command raced in the shared laptop target dir, and the run whose output I used did not cover the lib binary, while the other aborted. Corrected, sequentially verified evidence for PR fix(id/world): track retained admin-history bytes incrementally (#4633) #4747:
- lib world::tests module (the changed code's tests): 160 passed, 0 failed, including the 4 new admin_history_byte_ledger tests (each also verified by name).
- The full world filter cannot complete on the laptop: three CLI-args tests stack-overflow and abort the lib binary — cli::tests::module_trust_keys_are_repeatable_and_require_world_mode, cli::tests::test_cli_parse_world_new_and_stats, cli::world_limits_args_tests::flag_defaults_match_the_built_in_limits. Proven pre-existing on clean pre-merge main d05c22e (first parent of the PR fix(id/world): track retained admin-history bytes incrementally (#4633) #4747 merge), unrelated to this change; filed as id(cli): module_trust_keys_are_repeatable_and_require_world_mode stack-overflows, aborting the pkgs/id lib test binary #4751.
PR fix(id/world): track retained admin-history bytes incrementally (#4633) #4747's merge stands: the change's own tests pass, Codex review was clean, and the aborting tests fail identically without the change.

Navigation Menu

Repository navigation

id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633

id(world): `admin_history_bytes` re-serializes the entire retained admin history on every accepted event — track retained bytes incrementally

Evidence (all line numbers on PR head `47937fe3`)

Why it matters

Acceptance criteria

40 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event (PR #3998) — track retained bytes incrementally #4633

Description

id(world): admin_history_bytes re-serializes the entire retained admin history on every accepted event — track retained bytes incrementally

Evidence (all line numbers on PR head 47937fe3)

Why it matters

Acceptance criteria

Activity

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

40 remaining items

dezren39 commented on Nov 3, 2026

dezren39 commented on Nov 3, 2026

dezren39 commented on Nov 3, 2026

dezren39 commented on Nov 3, 2026

dezren39 commented on Nov 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

id(world): `admin_history_bytes` re-serializes the entire retained admin history on every accepted event — track retained bytes incrementally

Evidence (all line numbers on PR head `47937fe3`)