Skip to content

id(ci): F11.8 benchmark harness with committed baselines #539

Closed

Description

@dezren39

What and why

Benchmarks for guest replay of N events, journal append+fsync, checkpoint compaction and session fan-out to K clients, with committed baselines and a non-blocking CI job reporting >20% regressions. Several Phase 11/12 exits (fast CI lane, many worlds per node, fair scheduling) are stated as numbers, and nothing measures them today.

Source

Current status

Not started. No pkgs/id/benches directory; no bench/criterion in pkgs/id/Cargo.toml, justfile, flake.nix or .github/workflows/id*.yml (grep).

Acceptance criteria

  • Benchmarks exist for guest replay of N events, journal append+fsync, checkpoint compaction and session fan-out to K clients
  • Baselines are committed to the repo
  • A non-blocking CI job runs the benchmarks and reports any regression over 20% against the baseline
  • A just recipe and matching nix run .#<cmd> app run them locally (pkgs/id/AGENTS.md keeps the two in sync)

Size

L

Prerequisites

None.

Activity

  1. added
    kind/performancePerformance problem or optimization
    source/humanFiled directly by a human
    priority/normalOrdinary priority (the default)
    effort/lLarge: multi-day or open-ended; expect staged check-ins
    app/idPackage/app: id (pkgs/ or apps/ in this repo)
    area/ciCI/CD pipelines, workflows, and build automation (area)
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    and removed
    enhancementNew feature or request
    on Oct 26, 2026
  2. dezren39 commented on Oct 26, 2026

    @dezren39
    MemberAuthor

    Intake review

    Verified still live on main 14dd66b6 (2026-10-26, fleet monitor). There is no pkgs/id/benches directory and no bench/criterion reference in pkgs/id/Cargo.toml — no benchmark harness or committed baselines exist yet.

    Related: prerequisite for #551 (fast CI lane exit is a number this harness would help prove); its measurements also underpin #557 (many worlds per node) and the fair-scheduling numbers in Phase 12.

    Risks / suggested approach: the body's Size is L and the scope is four distinct benchmarks (guest replay, journal append+fsync, checkpoint compaction, session fan-out) — consider landing them as separable benches behind one harness so partial progress is committable. Keep the CI job non-blocking as specified, reporting >20% regressions against the committed baselines, and keep the just recipe and nix run app in sync per pkgs/id/AGENTS.md.

  3. added
    priority/highImportant and time-sensitive
    nowOrg Project gate: active priority — orchestrator adds to the board
    on Oct 27, 2026
  4. 110 remaining items

  5. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Oct 27, 2026
  6. dezren39 commented on Oct 27, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  7. dezren39 commented on Oct 27, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/idPackage/app: id (pkgs/ or apps/ in this repo)area/ciCI/CD pipelines, workflows, and build automation (area)effort/lLarge: multi-day or open-ended; expect staged check-inskind/performancePerformance problem or optimizationmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addednowOrg Project gate: active priority — orchestrator adds to the boardpriority/highImportant and time-sensitivesource/humanFiled directly by a humanstatus:done

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions