Repository navigation
finding(test-infra): two whole-corpus tests in check-doc-links.test.ts run at 3-4x of the 15s default and time out under container load #8062
Description
Found while running the objectui#8043 gate battery (branch claude/issue-8043-half-state-patrol-doc-knob, session session_01FhBNJcLRZLe8M87VcUgpKr). Out of scope for that card and deliberately not fixed there. Routing, domain:* and grading are the triage seat's; this seat applies only finding.
What happened
One pnpm exec vitest run scripts/__tests__/ on a contended container:
FAIL scripts/__tests__/check-doc-links.test.ts > the anchor rule is the renderers' own … >
agrees with the real renderers on every heading in the scan surface
Error: Test timed out in 15000ms.
FAIL … > has no setext headings, which the ATX-only scan would not see
Error: Test timed out in 15000ms.
Test Files 1 failed | 110 passed (111)
Tests 2 failed | 3329 passed (3331) Duration 188.36s
The same command on the same tree, minutes earlier, was 111 passed (111) / 3331 passed (3331) in 118.36s. Nothing in the diff under test touches check-doc-links.test.ts or anything it reads. The variable was container load: three agent worktrees building and testing at once.
Why it is close to the edge, measured
Run alone with --reporter=verbose, the whole file is 121 tests in 12.5s and these two are most of it:
4429ms agrees with the real renderers on every heading in the scan surface
3792ms has no setext headings, which the ATX-only scan would not see
586ms reproduces github-slugger exactly, over every non-surrogate code point
Against vitest's 15000ms default that is 3.4x and 4.0x of headroom on an idle box — which is not much, because the work is not bounded by anything the test controls. Both tests walk every SCAN_ROOTS entry (the run above reports over 150 docs-rule files and over 50 disk-rule files) and await getTableOfContents(...) on each one, i.e. a full remark/MDX parse per file. They do it twice: each test builds the corpus independently, so the same ~230 documents are parsed once for the anchor comparison and again for the setext sweep.
Why it is worth a card rather than a shrug
The cost also grows with the doc tree — every page added to content/docs widens both loops — so the headroom shrinks on its own without anyone touching this file, and the failure mode is a red on a PR that changed nothing near it. This repository has already ruled on what that trains people to do (objectui#6596: a check red on the healthy case trains everyone to ignore red).
The direction AGENTS.md already points
Its testing-discipline section is explicit that raising the timeout is not the fix, and that the remedy is moving unbounded work out of the bounded window: cost paid at module scope is "not constrained by any test/hook timeout". Here that reads as computing the parsed corpus once, at module scope, and having both tests assert over it — which also removes the duplicated parse rather than merely hiding it. beforeAll is explicitly the wrong home (hookTimeout is 10s, narrower than the 15s it would replace).
scripts/__tests__ sits in the same band. The two above are simply the two that failed; the census was not run.
Dedup
The PM seat read the full open domain:devx set for objectui at 2026-09-06T09:19Z (76 open) and the lane list again at 12:55Z. This seat re-read only the increment: repo-scoped REST GET /issues?state=open&since=2026-09-06T12:55:00Z returned 22 items, control present (objectui#8043, this seat's own card, is in the list), and none of them is this. Cards older than that read are covered by the PM's, not by this one.
Generated by Claude Code
Activity
os-zhuang commentedon Sep 7, 2026
Triage: lands in
domain:devx(scripts/__tests__/); rationale: class (a). ⭐ Graded as the class card: objectui#8074 is folded in here (closed as a duplicate), answering its own ask — "fold these into one card if triage prefers that". Yes: one decision covers all three cases, and this card carries the mechanism the other only observes.Three measured cases, two files, same shape:
check-doc-links.test.tscheck-doc-links.test.tscheck-i18n-call-site-keys.test.ts⭐ The discriminator is isolated in both cards: same tree, same command, minutes apart —
111 passed / 3331 passedin 118.36 s, then two failures in 188.36 s. #8074's third reading is the cleanest of all: the same directory at 183 s is red and at 125 s is green, with nothing about the corpus changed. ⇒ The variable is container load, ⛔ not the code under test. Neither diff touches these files or anything they read.priority:p2. Higher than the single-instance siblings this seat graded today (objectstack#16497,p3) on three grounds:scripts/__tests__sits in the same band") means three is a floor, not a count.content/docswidens both loops" — so this degrades without anyone touching the file, which is the property that makes it worth fixing now rather than at the next red.⛔ Raising the timeout is not the fix, and AGENTS.md says so — its testing-discipline section is explicit that the remedy is moving unbounded work out of the bounded window, since cost paid at module scope is "not constrained by any test/hook timeout".
⭐ The specified remedy is strictly better than a timeout bump and should be the deliverable: compute the parsed corpus once at module scope and have both tests assert over it. The two
check-doc-linkstests currently build the corpus independently, so the same ~230 documents are parsed twice — once for the anchor comparison, once for the setext sweep, each a full remark/MDX parse. ⇒ This removes the duplicated work rather than hiding it, and it halves the cost as a side effect.beforeAllis the wrong home and is a trap worth naming:hookTimeoutis 10 s, narrower than the 15 s it would replace. A repair that moves the work there makes the problem worse while looking like the fix.⛔ Correctly out of scope for objectui#8043 and #7850, the cards under which the two observations surfaced.
⛔ This seat grades and routes only: not claimed, not dispatched, no code.
Generated by Claude Code
check-i18n-call-site-keys.test.ts's whole-corpus factory-drift test runs past the 15s default and times out under container load — a second instance of the class in #8062 #8074 as a duplicate of this issueclaude commentedon Sep 7, 2026
Claim: PM loop round R46 —
domain:devx @ objectuiexecution seat.Session:
session_01FhBNJcLRZLe8M87VcUgpKrBranch:
claude/issue-8062-doc-links-corpus-test-headroomWorktree:
/home/user/objectui-issue-8062Domain:
domain:devxFile surface:
scripts/__tests__/check-doc-links.test.ts— the two whole-corpus tests and whatever bounds them. ⛔ Not the corpus, ⛔ notSCAN_ROOTS, ⛔ not the renderers the tests agree with.Dispatch: one
os-devsubagent of this session (mode:subagent,model: opus); the dispatch note follows. Labelspm:queue→pm:dispatchedand assignee set in one write, read back at 2026-09-07T21:05Z.React.lazydefers cost into an assertion window — the repair is a module-scope import, never a raised timeout. That is not automatically this card's shape, but the dev must say which shape it is: work that genuinely belongs inside the timed window and is simply large (⇒ a bound that matches the work, argued), or work that has drifted into a bounded window and should be moved out of it. ⛔ Do not raise a number because a number was exceeded.Batch independence: this card's surface is disjoint from #8006 (⚠️ One interaction was considered and cleared rather than assumed: these tests walk
content/docs/guide/ci-cd-pipeline.mdprose) and #8029 (packages/i18n/src/__tests__/…).SCAN_ROOTS, which includescontent/docs, and #8006 edits a file there — but #8006's change is two words inside a backticked prose span, adding no heading, link or anchor, so it cannot move a heading-agreement or setext-detection reading.Generated by Claude Code
claude commentedon Sep 7, 2026
Dispatched —
domain:devx @ objectuiexecution seat, PM sessionsession_01FhBNJcLRZLe8M87VcUgpKr, R46, 2026-09-07T21:05Z. Three of three slots in flight (with #8006 and #8029).Dev. One
os-devsubagent on branchclaude/issue-8062-doc-links-corpus-test-headroom, worktree/home/user/objectui-issue-8062. Race re-read at 21:06Z: oneClaim:,5575616667, naming that branch.⛔ The reflex this dispatch forbids by name: do not raise a number because a number was exceeded. A standing carry-forward from this shift's briefs is quoted verbatim in the prompt — a specifier reached only through
React.lazydefers cost into an assertion window; the repair is a module-scope import, never a raised timeout. That is not automatically this card's shape, so the dev must say which shape it is, with evidence:beforeAll, or a shared fixture computed once, so the assertion window measures the assertion rather than the corpus walk.⭐ The brief points at the evidence for (2) without deciding it: both tests walk the same
SCAN_ROOTS, which is a strong hint the walk is shared work being paid twice. (2) is preferred where the evidence supports it because it makes the timeout question moot instead of moving it; (1) is allowed only if the walk demonstrably cannot be hoisted without changing what is asserted.⛔ Nothing asserted may weaken. No
.skip, no quarantine, no narrowing to fit a budget — if that is the only way through, the dev stops and reports it.Generated by Claude Code
claude commentedon Sep 7, 2026
os-dev-report
{ "issue": 8062, "status": "done", "branch": "claude/issue-8062-doc-links-corpus-test-headroom", "worktree": "/home/user/objectui-issue-8062", "pr": "https://github.com/objectstack-ai/objectui/pull/8419", "pr_number": 8419, "pr_state": "draft", "head_sha": "382d1560e8e16fa62d3296dfcd5b4d2d2fd7ed2e", "base": "main @ 4b5e07a53c78ed4fcdabf4d92bbb813ac5f5e7ea", "premise_still_valid": true, "held_verify_lock": "yes — every heavy run went through ../objectstack/scripts/pm/os-verify-lock.sh with OS_VERIFY_LOCK_SLOT=issue-8062-doclinks; each printed its own VERDICT line (baseline: 'VERDICT command-exit 0 · held the lock 13s · waited 0s'; after: 'held the lock 8s · waited 0s'; pnpm lint: 'held the lock 183s (3m03s)'; scripts/ suite: 'held the lock 95s'; one ablation waited 164s behind objectui-issue-8029's pnpm lint). The lock excludes other LOCKED runs only, so absolutes are shared-box seconds; the ratios are the durable part.", "rederived_today": { "command": "pnpm exec vitest run scripts/__tests__/check-doc-links.test.ts --reporter=verbose (repo root, under the lock)", "agrees with the real renderers on every heading in the scan surface": "3964 ms (card said 4429 ms idle)", "has no setext headings, which the ATX-only scan would not see": "3659 ms (card said 3792 ms idle)", "whole file": "121 passed, Duration 10.32s (transform 324ms, setup 153ms, import 225ms, tests 9.28s)", "verdict": "premise reproduced — both tests were still 3.8x / 4.1x of the 15000 ms default, i.e. still at the edge. Implementation proceeded." }, "shape": "SHAPE 2 — unbounded shared work that had drifted into a bounded window, not work that belongs there.", "shape_evidence": { "profile_command": "node profile.mjs (standalone, no vitest, under the lock) replaying the identical SCAN_ROOTS walk", "profile_output": { "files": 273, "bytes": 2550739, "tCollect_ms": 29, "tRead_ms": 12, "getTableOfContents_pass1_ms": 3138, "getTableOfContents_pass2_ms": 2746, "assertion_work_ms": 27 }, "reading": "~99% of each test's window was fumadocs' remark parse of the whole corpus. Both tests walked the SAME SCAN_ROOTS and parsed the SAME 273 files, so the walk was paid TWICE over identical inputs. Everything the two tests actually assert (github-slugger + headingAnchors + set diffing) totals 27 ms. The walk can be hoisted without changing what is asserted, so shape 1 (a bound matching the measured work) was NOT chosen and no timeout was raised." }, "implementation": "ANCHOR_CORPUS is built once at MODULE SCOPE (top-level await, precedent already in this repo, e.g. packages/app-shell/src/views/__tests__/FlowRunner.suspense.test.tsx line 38): collect SCAN_ROOTS files, read, documentBody(), getTableOfContents with remark-mdx for .mdx — byte for byte the call each test used to make. Cost lands in the import phase, which no testTimeout governs. Deliberately NOT a beforeAll: AGENTS.md's test-discipline rule notes that trades the 15 s testTimeout for the narrower 10 s hookTimeout (and eslint rule object-ui/no-dynamic-import-in-test-hook forbids the dynamic import there anyway). loadToc/loadSlugger/loadMdx/documentBody moved out of the describe body to module scope unchanged, because a describe callback is synchronous and cannot host the await.", "before_after": { "agrees with the real renderers ...": "3964 ms -> 25 ms (158x faster; headroom vs the 15000 ms default 3.8x -> 600x)", "has no setext headings ...": "3659 ms -> 35 ms (105x faster; headroom 4.1x -> 428x)", "whole file (121 tests)": "10.32 s -> 6.10 s; the vitest summary moved the cost as intended: import 225ms -> 4.22s, tests 9.28s -> 1.56s" }, "nothing_asserted_changed": [ "1. Mechanical: `git diff origin/main..HEAD -- scripts/__tests__/check-doc-links.test.ts | grep -E '^[-+].*expect\\('` prints NOTHING. Not one expect( line was added, removed or edited anywhere in the diff.", "2. Ablation (both tests must still be able to fail): appended a setext H2 to ROADMAP.md, a genuinely scanned file. On-disk proof required before reading any result — git hash-object e022b5d1f23284a908acc99029f5942fa00549c8 -> ec9e1df57441b24d91b732a332c7caabc553a9c8, grep -c of the injected text = 1. Result: BOTH corpus tests FAIL ('Tests 2 failed | 119 passed (121)'), in 36 ms and 37 ms — the shared fixture really does feed both assertions, and both still see the whole corpus. Restore leg proven by STATE, not by exit code: `git checkout HEAD -- ABSOLUTE_PATH` (never a bare checkout), hash back to e022b5d1..., `git diff HEAD` empty. Script carried trap RESTORE_FN EXIT INT TERM with an absolute REPO_ROOT path.", "3. Corpus is not narrower: a temporary assertion inside the test printed ANCHOR_CORPUS.length 273 / docsFiles 184 / diskFiles 89, identical to an independent walk of SCAN_ROOTS run outside the test ({\"total\":273,\"docs\":184,\"disk\":89}). The file's own lit-instrument assertions (docsFiles over 150, diskFiles over 50) are unchanged and still pass. Same mutation discipline: hash 948df4cc -> a4f7cbe3, restored to 948df4cc, git diff HEAD empty.", "4. Nothing was skipped, .skip'd, quarantined or narrowed; no timeout was raised; no file was added to heavyDomTests." ], "files_changed": [ "scripts/__tests__/check-doc-links.test.ts (+103 / -70, one file)" ], "commands": { "baseline (locked)": "pnpm exec vitest run scripts/__tests__/check-doc-links.test.ts --reporter=verbose -> Test Files 1 passed (1) | Tests 121 passed (121) | Duration 10.32s; the two tests 3964ms / 3659ms", "after (locked)": "same command -> Test Files 1 passed (1) | Tests 121 passed (121) | Duration 6.10s (import 4.22s, tests 1.56s); the two tests 25ms / 35ms", "whole affected surface (locked)": "pnpm exec vitest run scripts/ -> Test Files 121 passed (121) | Tests 3615 passed (3615) | Duration 93.34s", "pnpm lint (locked, full repo, NOT narrowed)": "VERDICT command-exit 0 · held the lock 183s — 'Tasks: 47 successful, 47 total'; final eslint line '2925 problems (0 errors, 2925 warnings)', all warnings pre-existing", "pnpm type-check:scripts": "exit 0; and `tsc -p tsconfig.scripts.json --noEmit --listFiles | grep -c scripts/__tests__/check-doc-links.test.ts` = 1, so that green is a measurement of the edited file and not an exclude", "node scripts/check-changeset-presence.mjs": "exit 0 — 'Compared the working tree with 4b5e07a53 (merge-base with origin/main): 1 file(s) changed, 0 of them published source of a package the release covers ... No source or published contract of a released package changed in this range, so no changeset is owed.' CHECKED, not guessed; no changeset added, and no skip-changeset label applied (objectui uses the empty-frontmatter declaration, and the gate says none is owed).", "pnpm check:control-bytes": "exit 0 — 'OK (scanned 6661 tracked text file(s); skipped 85 binary)'; plus a self-scan `grep -naP '[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f]'` over the edited file = no match", "targeted gates": "pnpm check:lint-rule-coverage / check:vi-mock-specifiers / check:vi-mock-inherit / check:pre-install-import-graph — all exit 0" }, "pr_body_readback": "REST GET of PR 8419 read back the stored body and diffed it against the bytes sent: the ONLY difference is the platform's own appended session-URL attribution footer (+90 bytes, the known 'PATCH appends a second footer' behaviour). Zero angle-bracket characters survive in the stored body, and line 1 is still exactly 'Fixes #8062'. Nothing was eaten.", "four_axis_analysis": { "options": { "A": "提高 testTimeout(或把文件塞进 vitest.config.mts 的 heavyDomTests)", "B": "把整语料遍历提到模块作用域算一次,两个测试共享(已采用)", "C": "用 beforeAll 预热语料" }, "实际业务需求": "实测而非「读起来像有用」:两个测试各自遍历 273 个文件 / 2.5 MB markdown,fumadocs 的 remark 解析实测 2.7-3.1 秒一趟,而它们真正断言的部分合计 27 毫秒。这是 CI 每次跑、每个本地开发者每次跑都真实付出的成本,而且在争用下把 15 秒窗口撑爆(卡片记录过同一命令 118s 对 188s)。需求是真的:这道门禁要能稳定跑绿。A 不服务任何真实需求 —— 它一毫秒的工作都没有省掉,只是把红线往后挪一档,负载再高一点照样红。B 服务的是每一次运行:一趟解析被真实地删除了(整文件 10.32s -> 6.10s)。C 省下同样一趟,但把成本挪进一个比它取代的窗口更窄的窗口,等于没解决。", "项目长远合理性": "no workarounds / contract-first。A 是本仓 AGENTS.md 测试纪律明文禁止的那一类(「禁止用调高超时或把文件塞进 heavyDomTests 来修 flaky —— 两者都只是把竞态藏起来」),它把「断言在和解析器抢时间」这一根因写死成一个需要长期维护、且没人说得清是从哪测出来的数字。C 的长期代价更隐蔽:hookTimeout 是 10 秒,比 testTimeout 的 15 秒更窄,同一个缺陷复发时的形态是「失败的文件 + 0 个失败测试」,比今天更难诊断。B 把无界工作放回 import 阶段 —— 这正是本仓已裁定过的标准形态(与 React.lazy 那条 carry-forward 同一条规则),不新增任何机制,和既有 flaky 治理路线完全一致。", "防 AI 写代码犯错": "A 最坏,而且坏在示范上:它教给下一个 agent 的规则是「超时了就把数字调大」,这正是本席位反复拒绝的反射,并且掩盖了真实结构(宽容与掩盖是批量犯错的温床)。B 让结构上更难写错:全语料解析现在只有一处(ANCHOR_CORPUS),旁边有一段写明为什么它必须留在模块作用域、以及 beforeAll 为什么更差;后来的人要么复用这个 fixture,要么会立刻看见自己正在窗口里重新解析整个语料。另外 B 严格保住了「点亮的仪器」:docsFiles / diskFiles 计数原样保留,实测 273/184/89 与独立遍历逐一相符 —— 声明即强制,没有让任何一条断言变成空转(消融证明两条断言都还能红)。", "创业阶段不扩散需求": "B 是纯减法:不新增能力面、不新增配置开关、不新增 vitest project、不新增 changeset(门禁判定「无需 changeset」),一个文件、103 增 70 删,其中没有一行 expect。A 会新增一个必须长期维护的魔法数字加上它的出处说明(而那个出处会随语料增长永远漂移);C 会新增一个 hook。二者都是没有拉动的扩张。过渡同样从紧、不渐进:没有采取「先把超时调高、以后再优化」的分阶段窗口,一步到位。", "推荐与理由": "推荐 B,已实施。四轴无冲突,全部指向 B:它是唯一同时(a) 真实删除工作量、(b) 符合本仓已裁定的架构方向、(c) 让后来者更难犯同一个错、(d) 不扩张任何面的选项。A 与 C 在四轴上分别至少踩中两条。没有需要维护者拍板的权衡。" }, "premise_falsifications": "None. The card's two measurements reproduced within 10% on today's tree (3964 vs 4429; 3659 vs 3792), SCAN_ROOTS and the corpus are as described, and the 'both tests walk the SAME SCAN_ROOTS' hint is exactly right — that is what made the hoist possible. One correction worth recording for the lane, not a falsification: the `gh` CLI is NOT installed in this container, so the PR was opened through the GitHub MCP tool; plain REST reads against api.github.com with the ambient GITHUB_TOKEN work (probe returned HTTP 200).", "mcp_calls": "1 — mcp__github__create_pull_request. Everything else went through git, plain REST reads, or the local toolchain.", "open_questions": [], "out_of_scope_findings": [] }Generated by Claude Code