Skip to content

id(world): records.read cap pages can exceed the result_event size cap — admitted 768 KiB pages are replaced by result_too_large above 64 KiB #4888

Closed

Description

@dezren39

Finding (from PR #4856 CodeRabbit review, dispositioned by task 4880)

records.read pages are admitted up to MAX_RECORDS_PAGE_BYTES (768 KiB), but cap results are delivered through result_event (pkgs/id/crates/id-core/src/world_caps.rs), which replaces any serialized result event larger than MAX_EVENT_BYTES (64 KiB) with a result_too_large error. A records page between 64 KiB and 768 KiB therefore passes the page check in execute_records_read (pkgs/id/src/world/actor.rs) yet can never reach the program as records.

Evidence

  • pkgs/id/crates/id-core/src/world_caps.rs: MAX_EVENT_BYTES = 64 * 1024; result_event returns the result_too_large error above that size; RECORDS_READ_BOUNDS (added by PR refactor(id): add static provider registry (slice 1 of #1061) #4856) reports ProviderBound::ResponseBytes(MAX_RECORDS_PAGE_BYTES).
  • pkgs/id/src/world/actor.rs: execute_records_read admits records until used + size > MAX_RECORDS_PAGE_BYTES (768 KiB).
  • Note MAX_RECORDS_PAGE_BYTES also bounds the session-protocol page path (pkgs/id/src/world_session/mod.rs), which is a different consumer from cap result events — the fix must distinguish the two budgets, not blindly shrink the shared constant.

Why this is a follow-up, not part of PR #4856

PR #4856 (issue #1061 slice 1) is a no-behavior-change registry refactor; bounding the records page changes runtime behavior and belongs in its own change with its own tests.

Acceptance

  • A records.read cap result can never be admitted at a size result_event will refuse: derive the cap-result page bound from the event limit minus the result envelope, or route the result so the admitted page is deliverable.
  • RECORDS_READ_BOUNDS reports the bound that actually applies to cap results.
  • Regression test: a records page near the bound is delivered as records (not replaced by result_too_large), and the session-protocol page budget is unchanged.

Activity

  1. added
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  2. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: reviewed (coverage loop-back)
    Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 7935 (triage, queued, priority 30) in repo-health-code-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
    Found via: coverage loop-back 2026-11-03

  3. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
  4. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — task 7935 picked up by lane code-k (repo-worker-code). Execution lane assigned: opencode — handed off to the opencode claimant, which will launch a bound session to triage issue #4888 at current main.

  5. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  6. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done (triaged: confirmed live on main)

    Verdict: CONFIRMED at main 67f97516c (checked 2026-11-03T14:37Z)

    The defect is unchanged since PR #4856 merged (a39baa65, 2026-11-03T12:40Z). No open PR or branch owns this fix; no held/Drew-owned artifact is involved.

    Evidence

    • pkgs/id/src/world/actor.rs:822 — execute_records_read admits records until used + size > MAX_RECORDS_PAGE_BYTES.
    • pkgs/id/crates/id-core/src/world_caps.rs:536 — MAX_RECORDS_PAGE_BYTES = 768 * 1024.
    • pkgs/id/src/world/actor.rs:604 — every cap result is delivered through world_caps::result_event.
    • pkgs/id/crates/id-core/src/world_caps.rs:71,746-762 — MAX_EVENT_BYTES = 64 * 1024; result_event replaces any serialized event larger than that with result_too_large.
    • pkgs/id/crates/id-core/src/world_caps.rs:288-291 — RECORDS_READ_BOUNDS advertises ProviderBound::ResponseBytes(MAX_RECORDS_PAGE_BYTES), i.e. 768 KiB, the bound the cap path cannot deliver.

    So a records.read page whose records total anywhere from 64 KiB to 768 KiB passes the page check and is then replaced wholesale by result_too_large: the program can never read a page in that band, and the manifest advertises a bound that is not honored.

    One extra wrinkle for the fix

    execute_records_read estimates each record as key.len() + serde_json::to_vec(value).len() + 8 (actor.rs:821), but the serialized item is {"key":"…","value":…} — about 20 bytes plus a separator, and key escaping can grow it further. With many small records the estimate can undercount by tens of KiB, so shrinking the constant to MAX_EVENT_BYTES minus the envelope is not by itself sufficient. The fix should count the exact serialized item bytes (or reserve explicit headroom) and keep the existing fail-closed bad_args behavior for a single record that cannot fit.

    Fix path (scoped, no blockers)

    1. Add a cap-result page budget derived from MAX_EVENT_BYTES minus the result_event envelope (id ≤ MAX_NAME_BYTES), e.g. a named constant in world_caps.rs next to MAX_RECORDS_PAGE_BYTES.
    2. Enforce it in actor.rs:execute_records_read and report it from RECORDS_READ_BOUNDS so the manifest and the code agree.
    3. Do not shrink MAX_RECORDS_PAGE_BYTES: the session protocol (pkgs/id/src/world_session/mod.rs:1762, used at :1792, :1873) and the web records feed (pkgs/id/src/web/world_ws.rs:168) are different consumers and must keep the 768 KiB page budget; the existing session-protocol test at world_session/tests.rs:1435 should keep passing.
    4. Regression tests: a page just under the new bound is delivered as records through after_change() (not result_too_large); a page over the bound paginates via next_after without truncation; RECORDS_READ_BOUNDS reports the enforced bound; the shared 768 KiB budget is asserted unchanged.

    Ordering / blockers

    None. The change is independent, touches one subsystem, and needs no decision or access beyond this repo. Follow-up task 8079 is queued at priority 40 with the scope, tests, and acceptance above.

  7. added
    queue/queuedHas a queued task in the Supabase agent queue
    and removed
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  8. 2 remaining items

  9. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
  10. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  11. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

    • Agent: queue worker (repo-worker-code), lane code-c shift
    • Session / ID: task 8079
    • Found via: queue claim · resume from reproduced checkpoint (branch queue/8079-4888-records-read-cap, laptop worktree ~/git/fleet/8079-4888-records-read-cap)
    • Task: Implement the cap-result page bound for records.read: derive the page budget from MAX_EVENT_BYTES minus the result envelope in id-core world_caps, enforce it in actor execute_records_read with accurate serialized-size accounting, and make RECORDS_READ_BOUNDS report the enforced bound. The 768 KiB session-protocol/web page budget stays unchanged.
    • Expectation: effort s/m — one PR; done = regression tests pass (page just under bound delivered in full, over-bound page paginates via next_after, single over-bound record fails closed as bad_args), 768 KiB budget asserted unchanged.
    • Links: branch queue/8079-4888-records-read-cap
  12. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

    • Agent: queue worker (repo-worker-code), lane code-c shift
    • Session / ID: task 8079
    • Found via: queue claim · resumed at the reproduced checkpoint (implementation commit 0d22749 already on branch queue/8079-4888-records-read-cap)
    • Task: records.read cap pages are now bounded by MAX_RECORDS_RESULT_PAGE_BYTES (MAX_EVENT_BYTES minus a 1024-byte result-event envelope allowance), entries are counted at true serialized size, and RECORDS_READ_BOUNDS reports the enforced bound; the 768 KiB session-protocol/web page budget is unchanged and test-asserted.
    • Expectation: met — PR fix(id): bound records.read cap pages by the result-event limit (#4888) #5092 merged (squash 54ceece); Codex review completed with zero findings; tests on the laptop: cargo test -p id-core world_caps 7 passed, cargo test -p id records 154 passed (main suite).
    • Links: PR fix(id): bound records.read cap pages by the result-event limit (#4888) #5092 · branch queue/8079-4888-records-read-cap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working (prefixed form of the classic bug label)source/creatorFiled by the new-issue-creator jobtask/small-fixQueue packet kind: bounded fix with reproduction and targeted tests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions