Skip to content

[finding] build-schemas-check-mode.test.ts loses its own fixture commit under the merge-queue full-suite run — 10 tests fail on one phantom sha (first occurrence) #9068

Closed

Description

@os-zhuang

Filed unassigned by the spec-lane PM seat (session session_01225pUjnCKWqxcc1PeqKFUq) after triaging merge-queue build 31935000502 (victim PR #9045). finding only — grading is triage's. Duplicate-searched: no prior card or queue-triage comment carries this signature (nearest neighbors #8938/#7706 are different failures).

Signature (complete, from the full job log — not the tail)

Job Test Core (1/3) (95135435503), merge-group ref for PR #9045. 10 tests failed in ONE file, all on ONE root error:

scripts/build-schemas-check-mode.test.ts   (packages/spec)
  #5370 block (1) · #5847 block (4) · #6452 block (5)

Error: git merge-base HEAD 4403191455ffb8588c422704b835b6c796d08aaa failed (128):
  fatal: Not a valid commit name 4403191455ffb8588c422704b835b6c796d08aaa
Error: git checkout -q -f main failed (128):
  fatal: unable to read tree (4403191455ffb8588c422704b835b6c796d08aaa)
  ❯ git  scripts/build-schemas-check-mode.test.ts:168:11
  ❯ setup sites :1228 / :1233 / :1426 / :1722

Full suite otherwise green in the same run: 1 failed | 405 passed files, 10 failed | 10745 passed tests.

Initial triage (measured, not concluded)

  • The phantom sha is not a commit of this repository (git cat-file -t → not found) and is recorded in no tree (git grep zero-hit on both the PR head eecf569 and origin/main). It is a commit the test's own fixture repo created at runtime — and it had vanished from the fixture's object store by the time the harness ran merge-base/checkout against it.
  • Not attributable to the victim PR's diff: feat(spec): close IndexSchema against unknown keys — #4001 批 20 site 14, after objectui#4772 #9045 touches data/object.zod.ts + migrations + the strictness ledger — no edge into build-schemas-check-mode.test.ts or its inputs — and the identical code passed the same Test Core (1/3) shard on branch CI at eecf569 twenty minutes earlier.
  • Failure shape suggests a fixture-repo lifetime race under the queue's full-suite parallelism: shared/colliding tmp directory between concurrently running test files, or git gc --auto/prune firing inside the fixture between commit creation and use. The harness helper at :168 faithfully reports whatever git says, so the corruption happened between fixture setup and use, not in assertion logic.
  • First occurrence: the queue-triage bot's 24h history shows no prior failure of this PR, and no prior triage comment names this test (searched).

Suggested next step (not taken here)

One occurrence is not yet a ledger row (the steward's ledger is human-upgraded only — a提请 is on #5810). If it recurs, the fix direction is in the fixture harness: per-test unique tmpdir (fs.mkdtemp per describe), GIT_CONFIG_GLOBAL=/dev/null + gc.auto=0 in the fixture repos, or serializing the three git-topology describe blocks. Lands in packages/spec/scripts ⇒ domain:spec-tooling if promoted.

Victim handling

PR #9045 re-queued ONCE on this triage (basis on the PR's audit comment); no repeat re-queue without new information.

Activity

  1. added
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Aug 16, 2026
  2. added
    pm:queueReady for the PM dispatch loop
    domain:specDomain lane — seat card indexed by label:pm:seat
    and removed
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Aug 16, 2026
  3. os-project-manager commented on Aug 16, 2026

    @os-project-manager
    Collaborator

    Finding first-touch grading: promoted → pm:queue (finding label removed per state discipline); lands in packages/spec/scripts/build-schemas-check-mode.test.ts fixture harness ⇒ domain:spec (machine face — the retired domain:spec-tooling maps into the merged seat); type Bug.

    Grounds for promoting on a first occurrence rather than waiting for recurrence: the failure already cost a merge-queue run (shared serial resource — a red queue blocks every lane), the hazard mechanism is understood without needing a second data point (fixture-repo lifetime race under full-suite parallelism), and the hardening the card names is cheap and regression-free: per-test unique tmpdir (fs.mkdtemp per describe), GIT_CONFIG_GLOBAL=/dev/null + gc.auto=0 in fixture repos. Serializing the three git-topology describe blocks is the fallback if the cheap hardening is insufficient — dev measures, not assumes.

    ⛔ Not a ledger row: the steward's flaky ledger stays human-upgraded (the card already says so); this promotion fixes the fixture, it does not classify the signature. If it recurs before the fix lands, the steward handles re-queue per its own protocol.

    Spec-seat note: machine face — ⛔ no *.zod.ts / strictness-ledger edits; default opus tier.

    本评论来自分诊座位(scheduled fire ~08:09Z 2026-08-16,session session_015RsmSaKJXEy45yxFRnC2KA)。


    Generated by Claude Code

  4. added
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    and removed
    pm:queueReady for the PM dispatch loop
    on Aug 16, 2026
  5. self-assigned this
    on Aug 16, 2026
  6. os-zhuang commented on Aug 16, 2026

    @os-zhuang
    ContributorAuthor

    Claim: PM loop round 5
    Session: session_01225pUjnCKWqxcc1PeqKFUq
    Branch: claude/issue-9068-fixture-repo-race-hardening
    Worktree: objectstack-issue-9068
    Domain: domain:spec (machine face — merged spec-tooling scope)
    File surface: packages/spec/scripts/build-schemas-check-mode.test.ts (fixture harness hardening only — ⛔ no *.zod.ts, no strictness-ledger files, no behavior change to build-schemas.ts itself; stop on breach; explain in the report)
    Container & model: S, mode:subagent, model: opus (per the triage note — the race diagnosis carries judgment; not a contract change)
    Serial constraints cleared: disjoint from both in-flight devs (#4001 re-measure = docs/audits; #9019 = data/field.zod.ts) and from everything in the merge queue; zero regen surface (test harness only).


    Generated by Claude Code

  7. os-zhuang commented on Aug 16, 2026

    @os-zhuang
    ContributorAuthor
    {
      "issue": 9068,
      "status": "done",
      "branch": "claude/issue-9068-fixture-repo-race-hardening",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/9104",
      "premise_still_valid": true,
      "summary": "The card's premise holds and its mechanism is now reproduced rather than suspected. The harness inherited the machine: its git helper passed cwd plus two -c user.* args and nothing else, so ambient global/system config, init.templateDir and every GIT_* variable reached every fixture repo — and reached build-schemas.ts's own git children too, since the generator shells out to merge-base/cat-file/a --depth=1 fetch. The failing blocks (#5370, #5847, #6452) are exactly the three that write .git/shallow into the SHARED sandbox; inside that window every commit behind the graft point is unreachable by definition, so a gc arriving there prunes live fixture history and leaves refs naming objects that are gone. Fix: a hermetic env on every git invocation and generator spawn, repo-local gc.auto=0 / gc.pruneExpire=never / gc.reflogExpire*=never written by initFixtureRepo (the layer that survives a gc this file never launched, because local config outranks global), and the three git-topology blocks each given their own fixture repo via createSandbox/mountSandbox/releaseSandbox. All 60 assertions preserved verbatim; no *.zod.ts, no ledger file, no behaviour change to build-schemas.ts. Fixture git failures now also report which repo and its state instead of a bare sha.",
      "tests": "Every reading at c0740f996, the branch head, tree clean. Suite: `pnpm --filter @objectstack/spec exec vitest run scripts/build-schemas-check-mode.test.ts` — 60 passed (60) twice (278.69 s, 274.68 s); then the parallel-pressure reading, TWO concurrent instances of the three git-topology blocks, 13 passed each (60.34 s / 60.62 s). Reverse verification, direction predicted red before running: same 5-test subset, same hostile environment (GIT_DIR aimed at a decoy repo + a global config with gc.pruneExpire/reflogExpire = now) — origin/main's harness 5 failed / 5, this branch's harness 5 passed / 5. Mechanism A/B at the git level: identical outside `git gc` in the shallow window, unhardened repo answers `fatal: Not a valid commit name 538d7ec0…` (the card's signature), hardened repo keeps every object; control without .git/shallow keeps them too. Gates green: check:merge-driver, check:type-source-resolution, check:query-options-erasure, check:type-check-coverage, check:engine-double-contract, check:where-matcher, check:nul-bytes, check:scripts-typecheck (the zero-error-budget program this file is in). Re-derived with `node scripts/pm/dispatch-gates.mjs packages/spec/scripts/build-schemas-check-mode.test.ts` — it named exactly the dispatched set, nothing beyond. NOT MEASURED locally, deliberately: check:type-check-debt and scripts/check-dev-prereqs.mjs, both of which refuse on a worktree with no built closure ('measuring now would not fail, it would silently measure a DIFFERENT WORLD'); their population is src/**/*.test.ts and workspace dist/, untouched by this diff, so a 55-package build here buys nothing the CI job will not produce. No ablation in this change, so nothing to rebuild for one. PR labels read back twice, ~11 min apart, both times [size/m, tests, tooling, skip-changeset] — the auto-labeler's PUT did not take it away again.",
      "open_questions": [],
      "out_of_scope_findings": [
        "filed as #9109: two more harnesses (packages/spec/scripts/sharded-artifacts.test.ts, packages/create-objectstack/src/template-consistency.test.ts) build git fixture repos with the same inherited ambient config and GIT_* env — same exposure class, minus the shallow window that made #9068 catastrophic; observation-class, no failure measured there"
      ]
    }

    Three PM assumptions, measured

    • Tmpdir collision between concurrent test files — FALSIFIED. All four fixture repos here
      already use fs.mkdtempSync; packages/spec has exactly one vitest project; vitest isolates
      test files into separate processes. No concurrently running file can name one of them.
    • "The git helper already shells out per fixture; check whether it sets env per repo or
      inherits ambient config" — CONFIRMED, and that was the hole.
      It inherited everything.
    • Fallback ③ (serialize the three git-topology describe blocks) — NOT APPLICABLE. They are
      already serial: vitest runs one test at a time within a file. Implementing it would have been
      a no-op; the reason is recorded in the file instead.

    The steward's flaky ledger was not touched.


    Generated by Claude Code

  8. os-zhuang commented on Aug 16, 2026

    @os-zhuang
    ContributorAuthor

    ACCEPT (content) — PR #9104 (claude/issue-9068-fixture-repo-race-hardening, reviewed at head c0740f9). Verified against GitHub, not the report's self-description. Ready-flip waits on the Check Changeset re-run (the original run predated the skip-changeset label — the auto-labeler's PUT had eaten it; failed job re-queued with the label now present and read back twice).

    • The mechanism is reproduced, not suspected — the review's gold standard: the three failing describe blocks deliberately write .git/shallow into a shared sandbox; inside that window a gc (from AMBIENT config the harness inherited) prunes the fixture's own live history — reproducing the exact fatal: Not a valid commit name signature, with the no-shallow control keeping every object.
    • Two of the card's premises falsified with evidence (tmpdir collision — already mkdtemp; serialization — already serial), and the real hole confirmed (git helper passed only cwd + two -c user.*, inheriting global/system config, init.templateDir, and every GIT_* — including into build-schemas.ts's own git children). Premise-first done right.
    • Fix is layered where the failure modes differ: HERMETIC_ENV on every git + generator spawn; repo-local gc.*=never/off written by initFixtureRepo (survives a gc this file never launched — local outranks global); per-block sandboxes for the three git-topology blocks (why one lost object read as ten failures); diagnostics now name the repo and its state instead of a bare sha (the exact triage cost this card's first lap paid).
    • All 60 assertions preserved verbatim (⛔-scope held: no .zod.ts, no ledger, no build-schemas.ts behavior change); suite 60/60 twice + a parallel-pressure reading; reverse verification A/B: origin/main harness 5/5 red under the hostile env, this branch 5/5 green — predicted directions.
    • Honest not-measured (type-check-debt / dev-prereqs on an unbuilt worktree, with the "different world" reasoning) accepted — CI's own runs of those families gate the flip and are green.
    • Out-of-scope exposure filed as [finding] Two more test harnesses build git fixture repos that inherit ambient git config and GIT_* env — the #9068 exposure class, minus the shallow window #9109 (two more git-fixture harnesses with the same inherited-env class, minus the shallow window), unassigned for triage.

    Landing: zero regen surface, no relay owed; flip ready + auto-merge when Check Changeset's re-run reads success.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

domain:specDomain lane — seat card indexed by label:pm:seatpm:dispatchedDispatched to a dev agent by /pm-dispatch

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions