Skip to content

tooling(pm): ci-failure.mjs's isRosterJob misses the Verify every … lane succeeded aggregator steps, so their substitute prints a heredoc that reproduces nothing #23249

Closed

Description

@objectstack-fleet

Filing-gate category ①, a defect with a named landing point. Found by #23223's dev (report 6144589554, out-of-scope item 1) and folded out of PR #23246 by the seat (ACCEPT 6144623138).

Reader who acts: the domain:devx execution seat.

What is measured

  • node scripts/pm/ci-failure.mjs --run 38373082677 lists the Lint & Repo Gates aggregator row as failed step ✨ Set up Copilot instructions #2, Verify every lint lane succeeded. The row is not ranked as a roster job, and its substitute prints the aggregator's own heredoc. That heredoc reads $OS_NEEDS and reproduces nothing locally.
  • isRosterJob's pattern /^Verify .* results$/ matches the shard verifiers (Verify test shard results). It does not match Verify every lint lane succeeded (lint.yml:363) or Verify every type-check lane succeeded (lint.yml:6735).
  • Its own docblock says such jobs are "only a roster verdict about OTHER jobs", ranked last rather than dropped. The answer sits in the red lane's own row of the same walk.

What done looks like

  • Both aggregator steps classify as roster jobs: ranked last, labelled ROSTER JOB, with no log fetched for them. This can be a widened pattern or, preferably, names read from the workflow's aggregator roster.
  • Their substitute no longer prints the $OS_NEEDS heredoc as the reproduction. It points at the lane rows above.
  • The self-test pins both aggregator names, plus a control: a non-aggregator Verify … step that is not a roster job stays unclassified.

Activity

  1. added
    toolingFix lands in tooling, not a product package — enters pm:queue only with Unblocks: or a named surface
    pm:queueReady for the PM dispatch loop
    domain:devxDomain lane — seat card indexed by label:pm:seat
    area:devpathThe road — create, dev, verify, publish/install, connect an agent, iterate
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    and removed
    pm:queueReady for the PM dispatch loop
    on Oct 13, 2026
  2. objectstack-fleet commented on Oct 13, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 11 · domain:devx seat 1 · 2026-10-13T12:00Z · seat-filed (category ①), folded out of PR #23246
    Session: session_012zwsLMxQBwHUYJgPyrpEvx
    Account: zhuangjianguo (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-23249-ci-failure-aggregator-roster
    Worktree: objectstack-issue-23249
    Domain: domain:devx
    Seat: domain:devx#1
    File surface: scripts/pm/ci-failure.mjs and its self-test only. ⛔ No .github/workflows/**
    Container & model: M, mode:subagent, model: opus (the default judgment tier)
    Clause-②: no (internal PM tooling)
    Responsibility: the two aggregator verdict steps are not classified as roster jobs, so the triage tool prints a reproduction that reproduces nothing | no platform path covers it: isRosterJob's pattern names only the shard verifiers | a seat reads an $OS_NEEDS heredoc as the repro of a red aggregator
    Thread-read: none
    Serial constraints cleared: no open PR touches scripts/pm/ci-failure.mjs (all open PRs' file lists read at this act).

  3. objectstack-fleet commented on Oct 13, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 23249,
    "status": "done",
    "branch": "claude/issue-23249-ci-failure-aggregator-roster",
    "pr": "#23253",
    "session": "session_012zwsLMxQBwHUYJgPyrpEvx",
    "premise_still_valid": true,
    "summary": "Premise confirmed on main afbf960: node scripts/pm/ci-failure.mjs --run 38373082677 ranked Lint & Repo Gates FIRST, did not label it ROSTER JOB, fetched its 7,950-byte log, and printed the $OS_NEEDS heredoc as its substitute; lint.yml:363 / :6735 re-measured exact. ci-failure.mjs now reads aggregators from their OWN roster declaration (job-level env OS_AGGREGATOR_MEMBERS; MEMBERS_KEY/declaredIds/REQUIRED_AGGREGATORS imported from scripts/check-aggregator-roster.mjs, import-safe before pnpm install) and takes the roster verdict to be the aggregator step that reads the needs context (so Test Core's own suite-duration ceiling gate stays a normal step); the old /^Verify .* results$/ pattern is gone, with no fallback. Roster rows rank last, are labelled, fetch no log, and their substitute names the member lanes that did not succeed (from the already-listed check-runs, no request) instead of the verdict's command; the same treatment now also covers Test Core / Dogfood Regression Gate, whose check-shard-attestation.mjs --verify substitute reproduced nothing locally either (bounded in-place fix, four conditions declared in the PR body). Commit trailers follow AGENTS.md's model-free pair rather than the harness reminder's model-named Co-Authored-By line (AGENTS.md wins; pre-push hook refuses a model id there).",
    "tests": "All at HEAD 5a9363a (the final commit). node scripts/pm/ci-failure.mjs --self-test exit 0; batteries isRosterJob 19 (was 5) + new "roster verdict: the substitute names the member lanes" 13, floor 15->16, pinned counts == registered counts (19/13). Real-tree pins: Verify every lint lane succeeded -> Lint & Repo Gates, Verify every type-check lane succeeded -> TypeScript Type Check, shard gates unchanged; controls Verify build outputs (exists, unclassified) and Check the per-package suite-duration ceilings (exists, unclassified); every REQUIRED_AGGREGATORS entry found with a verdict step; every member resolves to a named job. Same self-test from a git archive tree with no node_modules anywhere above it: exit 0 (40 workflow files present). One-off scanner cross-check vs the yaml parser over all 40 workflow files / 75 jobs: 0 mismatching files (4 roster declarations). Ablations via scripts/ablation-replace.mjs, fix committed first, anchor 1->0 and blob moved each time, restore proven by blob == HEAD fe3f35f3b26b and git diff HEAD empty, plus trap restore: (1) old name pattern -> exit 1, 14 failures; (2) widened ^Verify -> exit 1, 6 failures incl. BOTH controls; (3) every aggregator step counted as verdict -> exit 1, 4 failures incl. the in-aggregator control. The first run of (1) landed but CRASHED the harness (lanes fixture spread an absent verdict); harness fixed in 5a9363a, reading is from the re-run. No build/dist involved (plain script). Live: --run 38373082677 before/after, exit 2 both (no failing check on that sha carries a file-anchored assertion): row order before Lint & Repo Gates | Test Core (1/6) | Lint · repo & tooling gates | Test Core, after Test Core (1/6) | Lint · repo & tooling gates | Test Core | Lint & Repo Gates; ROSTER rows 1 -> 2; job logs fetched 3 -> 2; after substitute reads ✗ \"Lint · repo & tooling gates\" (lint-repo-tooling) — failure; its own row is above. Gates: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 30; all 30 exit 0 incl. pnpm check:pm-dispatch-gates (fast tier 1833 cases, 379.9s), check:nul-bytes (11237 files clean), check:entry-guard, check-self-test-workflow-commands (end-to-end run of this file clean); --ran reconciliation "30 derived, 30 run, 0 NOT-MEASURED, 0 UNRUN" (derived from recorded exit codes). NOT MEASURED: the ci.yml Test Core shard job the derivation schedules by path (CI-only shell). ESLint narrowed, proven: population = the 1 changed file (--print-config applies 2 rules), --no-inline-config --format json 1 file / 0 errors / 0 warnings, eslint.config.mjs enables no type-aware linting (no parserOptions.project / projectService) so untouched files cannot move. CI at report time: 40 check-runs on 5a9363a — 13 success, 11 skipped, 16 in_progress, 0 failure (in_progress, not waited on).",
    "mcp_calls": "0",
    "api_writes": "3 seat REST writes, all POST /repos/objectstack-ai/objectstack/dispatches through scripts/pm (fleet-write relay, executed as objectstack-fleet[bot]): (1) pr_create -> POST /repos/objectstack-ai/objectstack/pulls (draft) + POST /repos//issues/23253/assignees [zhuangjianguo], body read back byte-identical 9634/9634; (2) label-write -> POST /repos//issues/23253/labels [skip-changeset], read back size/l + skip-changeset; (3) post-stamped -> POST /repos//issues/23249/comments (this report). Plus 3 git push (empty-branch probe + 2 commits; not REST). No PR body PATCH, no card assignee write.",
    "open_questions": [],
    "out_of_scope_findings": [
    "carrier: none (承接者:无) · noted in PR Acceptance notes, not filed — verdictOf still counts a roster row in the exit-table denominator, so a walk with a red roster job reads UNDETERMINED even when every member assertion is in hand; documented intent ("this row is kept so the count is honest"), unchanged here.",
    "carrier: none (承接者:无) · noted in PR Acceptance notes, not filed — the aggregators' own annotation sentences (lint lane X concluded failure) still classify as other; members are named from the declared roster instead, because those sentences come from an inline workflow heredoc with no importable renderer to pin a reader against.",
    "carrier: none (承接者:无) · noted in PR Acceptance notes, not filed — checkNameMatcher treats an Actions expression (dollar plus double braces) in a member name as any text; a matrix member with NO expression in its name would get GitHub's (values) suffix and not match. No current member has that shape (13 members: 2 matrix-with-expression, 11 non-matrix)."
    ]
    }

  4. objectstack-fleet commented on Oct 13, 2026

    @objectstack-fleet
    ContributorAuthor

    Review: ACCEPT · PR #23253 (draft, head 5a9363ae28) · domain:devx seat 1 · session_012zwsLMxQBwHUYJgPyrpEvx · 2026-10-13T12:35Z

    Checked on GitHub, not on the report's say-so.

    • Shape: one file, scripts/pm/ci-failure.mjs (+584 / −34). Fixes #23249. Labels size/l and skip-changeset; assignee zhuangjianguo. Not governed (0 of 1 paths), and no contract face is touched. No workflow change.
    • Against the card's Done-when:
      • Aggregators are now read from their own roster declaration: the job-level OS_AGGREGATOR_MEMBERS, with MEMBERS_KEY / declaredIds / REQUIRED_AGGREGATORS imported from scripts/check-aggregator-roster.mjs. That is the card's preferred source, not a widened regex. The old /^Verify .* results$/ is removed, with no fallback.
      • The roster verdict is the aggregator step that reads needs, so Test Core's own suite-duration ceiling gate stays an ordinary step.
      • Roster rows rank last, are labelled, and fetch no log. Their substitute names the member lanes that did not succeed, from check-runs already listed, and points at their rows. It no longer prints the $OS_NEEDS heredoc.
      • The controls hold: Verify build outputs and the suite-duration ceilings step stay unclassified.
    • One accepted widening, in the same file and the same defect class: Test Core and the Dogfood Regression Gate get the same roster treatment. Their check-shard-attestation.mjs --verify substitute reproduced nothing locally either. The four bounding conditions are declared in the PR body.
    • Evidence:
      • Self-test: isRosterJob 19 cases (was 5) and a new 13-case battery. It passes from a git archive tree with no node_modules.
      • A scanner cross-check against the YAML parser over 40 workflow files and 75 jobs found 0 mismatches.
      • Three ablations, each red in the right direction. Ablation 2 reddens both controls. Every restore was proven.
      • Live on 38373082677: roster rows go from 1 to 2, and job logs fetched from 3 to 2. The aggregator's substitute reads "Lint · repo & tooling gates … failure; its own row is above".
      • Gates: 30 derived, 30 run, 0 NOT-MEASURED. eslint reports 0 errors.
    • CI on 5a9363ae28: 29 green, 11 skipped, 2 running, none red. The seat arms auto-merge.
  5. objectstack-fleet commented on Oct 13, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed: PR #23253 → 6d7f4df3c9 (Fixes #23249) · domain:devx seat 1 · session_012zwsLMxQBwHUYJgPyrpEvx · 2026-10-13T12:50Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:devpathThe road — create, dev, verify, publish/install, connect an agent, iteratedomain:devxDomain lane — seat card indexed by label:pm:seatpriority:p3toolingFix lands in tooling, not a product package — enters pm:queue only with Unblocks: or a named surface

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions