check:console-injection silently drops its only tree-sensitive assertion when packages/spec is unbuilt — and nothing requires the spec blob

Observation found while measuring #9710 (which is the reason it is filed rather than fixed there — #9710's answer is a ruling, not a diff).

What I measured

scripts/check-console-injection.mjs makes five assertions that are pure functions of the restored dist + its stamp, and exactly one that reads this tree: the probe-expiry re-check

if (staleDetector && treeBlob && treeBlob.includes(staleDetector)) { … EXPIRED … }

treeBlob comes from readSpecBlob(specDir), which resolves packages/spec's exports map — i.e. it needs packages/spec/dist/**. When the spec is not built, readSpecBlob throws ProbeError, the script catches it, and the expiry branch is skipped because treeBlob is null. The run still exits 0, printing only:

ℹ Probe expiry not re-checked: this tree's spec at `…/packages/spec` has no built JavaScript to compare
  (build the spec — `pnpm --filter @objectstack/spec build` — to enable it)

Driving the real exported evaluate() against a fixed dist + stamp, varying only the tree:

tree state	verdict
spec not built	`PASS` — expiry check skipped
spec built, unchanged	`PASS`
spec built, moved forward	`PASS`
spec built, caught up to the published text	`FAIL(1)` — `✗ The stamped staleness probe … has EXPIRED.`

Row 1 and row 4 are the same tree question; only row 4 can ever be reached, and only with a built spec.

Why this is worth a card

--require-stamp exists precisely so the gate refuses to pass vacuously on a missing or unstamped dist. There is no equivalent for the spec side: no --require-spec-blob, and --require-stamp does not imply one. So the gate has a second vacuity path that its own design philosophy already rejects — the frozen-probe failure #8134 was filed about, one step further out.

It is not live today: ci.yml's Console Pin Gate runs pnpm exec turbo run build --filter=@objectstack/client... before the check, and @objectstack/spec is in that closure (verified: turbo … --dry=json lists 32 packages, spec among them). But nothing enforces that ordering — it is step adjacency, not a contract. Reordering the steps, or invoking the script from any job that does not build the closure, turns the expiry check off silently and still reports green.

Suggested shape (not prescriptive)

Either make --require-stamp imply a readable spec blob, or add an explicit flag and pass it in ci.yml, so a run that cannot perform the expiry re-check says so as a failure rather than an ℹ. A cheaper variant worth pricing: derive the expiry probe from packages/spec/src/** source text instead of built JS — .describe() arguments are plain string literals present in both — which would make the assertion buildless and remove the dependency entirely.

Refs: #9710 (where this was measured) · #9706 (the gate) · #9667 · #8134

docs(check-console-injection): record why packages/spec is not in ci.…

First-touch grading (triage seat): promoted to pm:queue, Task. Preferred shape is the card's cheaper variant if it prices out — derive the expiry probe from packages/spec/src/** source text, removing the built-spec dependency entirely; otherwise make --require-stamp imply a readable spec blob (or add the explicit flag and pass it in ci.yml) so a run that cannot re-check expiry fails instead of printing an ℹ. Not live today (verified reasoning in the body), so no priority label. Size/model suggestion: S–M, opus.

Generated by Claude Code

Claim: /pm-dispatch domain:devx execution seat (bug-first, batched round).
Session: session_01DdCnBGcHeufjrq7drTD3wt
Branch: claude/issue-10428-console-injection-spec-blob
Worktree: objectstack-issue-10428
Domain: domain:devx
File surface: scripts/check-console-injection.mjs + the Console Pin Gate step in .github/workflows/ci.yml

Note for the seat: ci.yml is in scope only for the Console Pin Gate step's flags. ⛔ Do not touch the changeset steps in any workflow (#9465 fence).

⛔ Governed surface is out of scope for this diff (docs/adr/**, .claude/**, skills/**, AGENTS.md, CLAUDE.md, content/docs/releases/**) — propose wording in the report instead. ⛔ #9465 epic fence also out of scope (.changeset/ tooling, cut-rc.yml, release.yml, root package.json); a .changeset/*.md entry for this change is expected and is NOT fenced.

Generated by Claude Code

{
  "issue": 10428,
  "status": "done",
  "branch": "claude/issue-10428-console-injection-spec-blob",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/10597",
  "premise_still_valid": true,
  "summary": "The card's defect reproduced exactly. Fixed by OPTION 1: --require-stamp now also requires that every stamped staleness probe was actually re-checked, so a run that cannot perform the expiry re-check fails instead of printing an info line. Scoped to a probe that EXISTS and went unexamined (new `expiryDeferred` counter), not to 'the spec is unbuilt' -- a no-skew stamp has no expiry question to skip and still passes. Exit 1, matching the other --require-stamp refusals (no dist, no stamp); 2 stays reserved for an unreadable tree. NO ci.yml CHANGE: the requirement rides the flag the gate already passes, so no call site can forget it -- an opt-in --require-spec-blob that a future job neglects to pass would reproduce the exact failure this card is about. The bare invocation is unchanged, so a bare checkout keeps the advisory notice. Option 3 was priced FIRST and declined on measurement (numbers below); the measurement is recorded in the script header so the next asker does not re-price it, and the header's #9710 ruling prose that my change falsified ('spec unbuilt PASSES') was corrected in the same diff. One file changed: scripts/check-console-injection.mjs (+172/-12).",
  "option_3_pricing": {
    "verdict": "DECLINED on measurement -- the brief's premise is ~98% true and the missing 2% falls on the silent-pass side",
    "method": "real describeCandidates()/readSpecBlob() over a freshly built packages/spec (dist blob 9,693,406 bytes; 595 non-test source files); numbers reproduced identically against an independently built spec in a sibling worktree, so this is structural, not dist/src skew",
    "dist_probe_candidates": 2993,
    "false_negative_channel": "44 of 2993 (1.5%) dist probe candidates appear NOWHERE in src/** as literal text; 59 (2.0%) against the narrower src/**/*.zod.ts subset the package actually publishes. A missed probe reads as 'not expired' = SILENT PASS, the same failure class the gate exists to end.",
    "false_positive_channel": "260 of 3161 (8.2%) src candidates are description text the built package does not ship; a detector matching one would report EXPIRED on a healthy tree (hard red).",
    "mechanism_1": "25/44 quote-and-escape normalisation -- source writes '...definition\\'s...', esbuild re-emits \"...definition's...\". Same characters, different bytes, so a literal substring search over source misses it. 20 of the 25 have their pre-quote prefix present in src, confirming the text IS there, spelled differently.",
    "mechanism_2": "19/44 constant-folded concatenation -- e.g. packages/spec/src/api/protocol.zod.ts:341-342 writes `'Owning package machine id. Present only when the resolved item declares ' + '`_packageId`.'`; the bundler folds it into one literal that exists in no source file.",
    "why_not_rescuable": "Symmetric src-vs-src derivation cancels both channels, but that is a redesign of what assert-console-spec-injection.mjs STAMPS (and of the round-trip self-test case), and it still cannot make the BUNDLE side buildless -- which is where the dist dependency actually lives. Enforcing the dependency was the cheaper correct move."
  },
  "four_row_table_before": [
    {
      "tree_state": "spec not built",
      "requireStamp_false": "PASS",
      "requireStamp_true": "PASS  <-- expiry check silently skipped"
    },
    {
      "tree_state": "spec built, unchanged",
      "requireStamp_false": "PASS",
      "requireStamp_true": "PASS"
    },
    {
      "tree_state": "spec built, moved forward",
      "requireStamp_false": "PASS",
      "requireStamp_true": "PASS"
    },
    {
      "tree_state": "spec built, caught up",
      "requireStamp_false": "FAIL(1) EXPIRED",
      "requireStamp_true": "FAIL(1) EXPIRED"
    }
  ],
  "four_row_table_after": [
    {
      "tree_state": "spec not built",
      "requireStamp_false": "PASS (advisory, unchanged)",
      "requireStamp_true": "FAIL(1) 'A stamped staleness probe was never re-checked for expiry.'  <-- CHANGED"
    },
    {
      "tree_state": "spec built, unchanged",
      "requireStamp_false": "PASS",
      "requireStamp_true": "PASS"
    },
    {
      "tree_state": "spec built, moved forward",
      "requireStamp_false": "PASS",
      "requireStamp_true": "PASS"
    },
    {
      "tree_state": "spec built, caught up",
      "requireStamp_false": "FAIL(1) EXPIRED",
      "requireStamp_true": "FAIL(1) EXPIRED"
    }
  ],
  "tests": "All gate results below are the gates' own printed verdict lines, captured with redirect-then-capture (never a bare $? after a pipe). Union re-derived and run AFTER the final commit, at HEAD 357f67864b (worktree clean against that commit).\nGATES (derived by `node scripts/pm/dispatch-gates.mjs` with no paths -- it took the change set from the merge base itself; 3 families matched, all green):\n  - pnpm check:console-injection -> exit 0; '<check> check-console-injection --self-test: 27 assertions over real fixture trees (real evaluate() path)' + 'No console dist at packages/console/dist -- skipping injection check'\n  - pnpm check:cross-package-test-inputs -> exit 0; 'All 60 self-test cases passed.' / 'OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.'\n  - node scripts/check-nul-bytes.mjs -> exit 0; 'OK (scanned 6187 text file(s) ... no raw ASCII control bytes)'; plus a direct control-byte self-scan of the changed file and of the PR body: 0 hits each.\n  - eslint scripts/check-console-injection.mjs --no-inline-config -> exit 0.\nFOUR-ROW TABLES: both driven through the REAL exported evaluate() with dist+stamp held fixed and only the tree varying, in BOTH flag modes (the card's table did not state a flag; row 1 passes in both modes before the fix, so the card's table holds either way). Row 1 changes verdict only under --require-stamp, which is what ci.yml runs; rows 2-4 unchanged in both modes.\nREJECT SIDE ASSERTED POSITIVELY: new self-test case 7b builds the unbuilt-spec tree (makeUnbuiltSpecPkg -- real manifest and exports map, no dist/) and asserts exit 1 plus branch-unique wording ('never re-checked for expiry', and that the message names the build that clears it). A 'no error' test would have proved nothing: before the fix that fixture exited 0 with an info line, which is itself a green asserting nothing.\nPOSITIVE CONTROL (two levels): (a) in-fixture -- built spec + --require-stamp still exits 0, without which case 7b would be satisfied by a script that fails --require-stamp unconditionally; (b) real-CLI -- `node scripts/check-console-injection.mjs --require-stamp --dist <fixture> --spec packages/spec` against the ACTUAL built spec (9.7 MB blob) exits 0, while the same dist+stamp with an unbuilt spec exits 1 with the new message. PRECISION CONTROL: a no-skew stamp on an unbuilt tree still exits 0, so the refusal cannot fire over an assertion nobody was owed.\nABLATION: no build/dist is involved -- these scripts run from source, nothing resolves through a dist/, so there is no rebuild leg to report. Predicted direction was declared BEFORE running: red with exactly 3 failures, positive control and precision case staying green. Mutation `if (expiryDeferred > 0)` -> `if (expiryDeferred > 0 && !requireStamp)`; CONFIRMED ON DISK before the run by grep -c of both the removed and the injected text (removed 1->0, injected 0->1) plus the git diff hunk -- not by the editor's exit code. Result matched the prediction exactly: exit 1, '3 failure(s)': 'unbuilt spec is fatal under --require-stamp: expected 1, got 0', 'skipped-expiry failure must say the probe was never re-checked', 'skipped-expiry failure must name the build that clears it'. RESTORE LEG run and confirmed the same way: mutation text absent (0), original present (1), worktree clean against the commit, self-test back to 27 assertions green.",
  "brief_corrections": [
    "OPTION 3's stated premise is FALSE at the margin that matters. The brief said '.describe() arguments are plain string literals present in both'. True for 98.5% of the probe pool; the 1.5% that is not (2.0% on the published .zod.ts subset) fails in the SILENT-PASS direction, which is the one direction this gate cannot tolerate. Two named bundler transforms, quoted in the PR body and the script header with a file:line example. So option 3 was rejected on evidence, exactly as the brief asked -- not waved off.",
    "The brief's fallback framing ('take option 1 or 2') presented them as near-equivalent. They are not: option 2 (--require-spec-blob passed in ci.yml) reproduces this card's own failure mode one level up -- an invariant a call site can forget, which is precisely the 'step adjacency, not a contract' problem being fixed. Option 1 was taken for that reason, and it needs no ci.yml edit at all, so the Console Pin Gate fence and the #9465 changeset fence are respected trivially.",
    "A CHANGESET IS THE WRONG ROUTE HERE, contradicting the brief's 'a .changeset/*.md entry is expected'. The diff is one repo-internal CI gate script; root package.json is private: true and no published package ships scripts/**, so it releases nothing. pr-automation.yml's changeset gate is binary (changeset OR skip-changeset label) with no path exemption, and its own error text names 'releases nothing' as the skip-changeset case while explicitly ruling out an empty-frontmatter changeset. Applied the skip-changeset label instead and CONFIRMED BY READ-BACK (labels now ['size/m','skip-changeset']; the additive POST preserved the bot's size/m). Flagged in open_questions so the PM can overrule.",
    "MY OWN ASSUMPTION, corrected before it reached the pricing: I expected the published spec tarball to ship dist only, which would have made src-based derivation structurally impossible on the vendored side. It actually ships src/**/*.zod.ts (package.json `files`). That does NOT rescue option 3 -- measured against exactly that subset the false-negative rate is 2.0%, slightly WORSE than against full src -- but the reason option 3 fails is the bundler transforms, not tarball contents.",
    "NOT VERIFIED EITHER WAY: the brief's note that `gh` is not installed. I never invoked it -- used the GitHub MCP tools and curl with $GITHUB_TOKEN throughout, as instructed -- so I can neither confirm nor contradict it.",
    "PLATFORM BEHAVIOUR, not another actor: GitHub appended a second '---' + session-URL signature footer to the PR body at creation. My body already ended in the session-URL form and nothing of mine was stripped (byte-level diff of sent vs stored shows an addition only). Left as-is rather than edited, since editing is the path that strips footers."
  ],
  "open_questions": [
    {
      "question": "Changeset route: the dispatch brief expected a .changeset/*.md entry; the repo's own changeset gate says a PR that releases nothing should take the skip-changeset label, and rejects empty-frontmatter changesets outright. I took the label. Overrule?",
      "options": [
        "A - keep skip-changeset (current state): the diff is one internal CI gate script, root package is private, no published package ships scripts/**, so nothing releases. Matches the gate's own error text. Also avoids pulling in the 5 extra changeset-triggered gate families that dispatch-gates.mjs reports would apply.",
        "B - add a changeset naming some package: would declare a release for a package whose shipped code did not change, and a non-empty changeset for a no-op release is the thing check-empty-changeset.mjs and #5471 exist to stop."
      ],
      "recommendation": "A, because the change ships in no published artifact -- the label is what the gate itself prescribes for that case, and B would fabricate a release. Reversible in one step if the PM disagrees: drop the label, add the changeset."
    }
  ],
  "out_of_scope_findings": [
    "filed as #10595: `check:console-injection --require-stamp` still passes on a stamp whose `packages` array is empty (readStamp accepts [], evaluate falls through to 'Nothing assertable in this stamp' and exits 0) -- the same 'satisfies the flag while asserting nothing' shape, one input over. Not reachable from the real producer (assert-console-spec-injection.mjs writes exactly one hard-coded entry at its single writeStamp call site, on both its skew and no-skew paths), and whether it should refuse is a judgement call rather than something existing evidence pins -- so it failed the in-scope-fix test and was filed unassigned with the `finding` label, not queued."
  ]
}

Generated by Claude Code

docs(check-console-injection): record why packages/spec is not in ci.…

Navigation Menu

Repository navigation

`check:console-injection` silently drops its only tree-sensitive assertion when `packages/spec` is unbuilt — and nothing requires the spec blob #10428

What I measured

Why this is worth a card

Suggested shape (not prescriptive)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

check:console-injection silently drops its only tree-sensitive assertion when packages/spec is unbuilt — and nothing requires the spec blob #10428

Description

What I measured

Why this is worth a card

Suggested shape (not prescriptive)

Activity

os-zhuang commented on Aug 21, 2026

claude commented on Aug 21, 2026

claude commented on Aug 21, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`check:console-injection` silently drops its only tree-sensitive assertion when `packages/spec` is unbuilt — and nothing requires the spec blob #10428