Skip to content

[finding] driver-sql's live PG + MySQL matrix runs under vitest's default 5000ms — the only live-DB driver in the repo with no testTimeout, and it just dequeued an unrelated PR #16434

Closed

Description

@zhuangjianguo

Filed by the domain:engine dispatching seat from a merge-queue failure that dequeued a changeset-body-only PR. ⛔ Observation, not a runtime defect in shipped code — no assignee, no pm:queue, no domain:*, no priority; triage grades it.

What happened

Queue build 34069423383, job Temporal Conformance (live PG + MySQL):

× §2 a millisecond-precision delta cursor does not SKIP the updated row 5004ms
FAIL  src/sql-driver-11224-update-stamp-precision.test.ts
  > #11224 — the UPDATE door stamps at the audit column's precision (live mysql)
  > §2 a millisecond-precision delta cursor does not SKIP the updated row
    ↳ Error: Test timed out in 5000ms.

PR #16430 was dequeued at 00:45:57Z with reason CI_FAILURE. That PR changes one file — .changeset/platform-iana-timezone-columns.md, +46 / −1 against merge base 0a038cc06 — and zero files under packages/drivers/driver-sql/. The queue runs the full suite, so it paid for a rebuild of everything behind it on a failure nothing in its own diff could cause.

The finding — this is a missing budget, not a flaky assertion

⭐ §2 has nothing to de-race. It deliberately does no sleeping. The file's own head docblock says so, in the section recording what #11572 traded away when it fixed §3:

What was traded away is the back-to-back WRITE RATE — §3 no longer fires […] §1, §1b and §2 still create and update with no sleep between them ("No sleep and no backdating", §1), so the same-millisecond path stays measured where it belongs.

So the two prior repairs to this file do not apply here: #11572 drove §3's clock instead of racing it, and #13691 moved §3's validity ceiling. Neither touches §2, and neither should — §2's subject is the unslept same-millisecond path.

What §2 actually spends is network time. With ROUNDS = 6, each iteration issues four live round-trips — driver.create → readAudit → driver.update → the cursor comparison, which is deliberately made by the server (knex(MANAGED).where('updated_at', '>=', before.rawCreatedAt).first(), so the comparison happens in the column's own type rather than in JS). That is 24 live round-trips inside one test.

And the test has no budget of its own:

probe reading
explicit per-test timeouts in sql-driver-11224-update-stamp-precision.test.ts 0
explicit timeouts across 20 sibling driver-sql live-cell suites 0
testTimeout in packages/drivers/driver-sql/vitest.config.ts absent ⇒ vitest's default 5000 ms

⭐ The firing control: six packages in this repo do set one, and the closest analogue sets 30 s

grep -rn "testTimeout" --include=vitest.config.ts (non-node_modules), complete:

package testTimeout
packages/drivers/driver-mongodb 30_000
packages/services/service-datasource 60_000
packages/spec 60_000
packages/qa/http-conformance 30_000
packages/plugins/plugin-auth 10_000
packages/metadata-fs 10_000
packages/drivers/driver-sql — none —

⇒ The one driver package whose suite talks to two live servers is the only live-database driver in the repo with no budget at all, while its nearest sibling — the other live-DB driver — carries 30 s. driver-sql's config is not silent on hard-won operational facts either: it already carries globalSetup: ['./src/live-dialect-matrix.globalsetup.ts'] for #9350 (each live file owns a schema on Postgres / a database on MySQL, and MySQL refuses the handshake for a database that does not exist). The timeout simply never got the same treatment.

⚠️ What this finding does NOT claim. It does not claim the default is wrong for the in-memory / SQLite cells — they are the majority of this package's tests and 5 s is a reasonable guard there. The gap is specific: the matrix multiplies every live file across cells and the live cells alone are exposed to network and server load inside a budget sized for neither.

Why it is worth a card rather than a re-run

A re-run is the correct response to this instance and it was spent (#16430 re-queued at 00:49:01Z). The card is about the class: a timeout in the merge queue does not merely redden one PR — it dequeues it and rebuilds everything behind it. The 24h ledger recorded 0 other queue failures, so this is the first, which is exactly when the budget question is cheap to answer.

Shape of the fix, for triage — direction only, ⛔ not a ruling

The obvious move is a testTimeout on packages/drivers/driver-sql/vitest.config.ts. Two things a dev should settle rather than assume:

  1. One budget or two? A single package-wide value raises the ceiling for the fast in-memory cells too, which weakens them as a guard. A per-cell budget (live cells only) keeps the fast cells strict, at the cost of the live matrix carrying it explicitly at each it.
  2. What number, and derived from what? ⛔ Not copied from driver-mongodb by analogy. The honest derivation is the observed live round-trip cost — the same discipline [finding] sql-driver-11224-update-stamp-precision §3 now reds on a SLOW runner too — the span < 1000ms line is a validity precondition, and #11572's fix spends deliberate clock time inside that same budget #13691 applied to §3's span ceiling, where a bound was derived from the conclusion it guards rather than picked.

Re-check

git show origin/main:packages/drivers/driver-sql/vitest.config.ts | grep -c testTimeout          # expect 0
grep -rn "testTimeout" --include=vitest.config.ts . | grep -v node_modules                        # the control table above
git show origin/main:packages/drivers/driver-sql/src/sql-driver-11224-update-stamp-precision.test.ts | grep -n "^const ROUNDS"

Readings taken on origin/main ac76425f0.

Refs: #16430 (the PR this dequeued) · #11224 (the defect the suite pins) · #11572 (§3 driven, not raced) · #13691 (§3's validity ceiling) · #9350 (the live-matrix globalSetup)

Activity

  1. added
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Sep 7, 2026
  2. claude commented on Sep 7, 2026

    @claude
    Contributor

    Second instance, in a different package — and it dequeued a second unrelated PR

    domain:engine dispatching seat, 2026-09-07T05:5xZ. ⛔ Still an observation, no grading, no domain:*, no priority — this is added because it widens the card's population, not because it changes its verdict.

    The new instance

    Queue CI run 34086147182, Test Core (4/6) — 17 jobs, exactly one failing test:

    FAIL  packages/plugins/plugin-email/src/email-service.queue-delivery.test.ts:279
      > EmailService — queue delivery on
      > still refuses the queue for attachments OVER the limit, and stores nothing (#5177)
        ↳ Error: Test timed out in 5000ms.
    

    It dequeued #16442, whose eight changed files include zero under packages/plugins/plugin-email/, and whose own PR CI is 41/41 with 0 failing on the same head.

    The same probe, the same answer, a different package

    probe driver-sql (this card, 05:4xZ) plugin-email (new)
    testTimeout in the package's vitest.config.ts absent absent
    explicit per-test timeouts in the failing file 0 0
    effective budget vitest default 5000 ms vitest default 5000 ms
    what it cost dequeued #16430 (a changeset-body-only PR) dequeued #16442 (a @objectstack/core narrowing)

    The control table from the original filing is unchanged and still fires: six packages in this repo do set testTimeout — driver-mongodb 30_000, service-datasource 60_000, spec 60_000, qa/http-conformance 30_000, plugin-auth 10_000, metadata-fs 10_000.

    ⚠️ What this does and does not change about the card

    Does: the population is wider than "the one live-DB driver". Two packages, two queue-only timeouts, two dequeued PRs, in one night — and neither failing test belongs to the PR it dequeued. Whatever the fix is, scoping it to driver-sql alone would leave the second instance standing.

    Does not: it does not make the two instances the same defect. driver-sql's is specific and derivable — 24 live round-trips to two real servers inside a 5 s budget. plugin-email's failing test is in-process and has no live server, so why it needs more than 5 s is a separate question this seat has not answered, and ⛔ raising its budget without answering it would convert a visible flake into a slow test nobody notices. That question belongs to whoever takes this card.

    ⚠️ ⛔ And the direction NOT to take by reflex: a repo-wide default is the tempting fix and it is the one that costs the most — it raises the ceiling for every fast in-process suite, which is where a 5 s guard is doing real work. The two derivations (what a live-server matrix needs; whether an in-process email test should need it at all) are separate, and the second one may end in "fix the test" rather than "raise the budget".


    Generated by Claude Code

  3. added
    toolingFix lands in tooling, not a product package — enters pm:queue only with Unblocks: or a named surface
    pm:queueReady for the PM dispatch loop
    domain:engineRetired 2026-10-22, alias of domain:server (engine surface); in-flight cards keep it until they land
    and removed
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Sep 7, 2026
  4. added theissue type on Sep 7, 2026
  5. os-zhuang commented on Sep 7, 2026

    @os-zhuang
    Contributor

    Triage: lands in domain:engine (packages/drivers/driver-sql); rationale: class (a) — ⭐ this one did not predict a cost, it measured one that was already paid. PR #16430 (+46 / −1 on a single .changeset/*.md, zero files under driver-sql/) was dequeued at 00:45:57Z with CI_FAILURE on a test its diff cannot reach, and the queue rebuilt everything behind it. Bug.

    priority:p2. Two facts set it there and both are on the card: the failure mode is a merge-queue dequeue, which is strictly more expensive than a red PR check, and the 24h ledger recorded 0 other queue failures — so this is the first instance, which is exactly when the budget question is cheap to answer rather than tangled with a backlog of re-runs.

    ⭐ The firing control is what makes this gradeable rather than arguable, and it should survive into the PR: six packages set testTimeout; driver-mongodb — the other live-DB driver — sets 30_000; driver-sql, the one package whose suite talks to two live servers, sets none and inherits vitest's 5000ms. A complete grep, a named comparator, and an explicit statement of what the finding does not claim (5s is fine for the in-memory / SQLite cells, which are the majority). That last fence is the difference between a budget card and a "raise the timeout" reflex.

    ⭐ This is a missing budget, not a flaky assertion, and the card proves it rather than asserting it: §2 deliberately does no sleeping — the file's own head docblock records that #11572 kept "§1, §1b and §2 … with no sleep between them" because the unslept same-millisecond path is §2's subject. What it spends is network: ROUNDS = 6 × four live round-trips = 24 live round-trips in one test, with the cursor comparison deliberately made server-side so it happens in the column's own type. ⇒ ⛔ Do not "fix" §2 by sleeping, backdating or driving its clock. The two prior repairs to this file (#11572 drove §3 instead of racing it; #13691 moved §3's validity ceiling) touch §3 only and ⛔ must not be extended to §2 — doing so would delete the thing the test measures.

    ⛔ Do not treat the next occurrence as a flake and re-run it. A re-run was the right response to this instance and it was already spent (#16430 re-queued at 00:49:01Z). The card is about the class, and the cause is named.

    Route — the two questions the card says a dev must settle rather than assume, ratified here as blocking, not optional:

    1. One budget or two. A package-wide testTimeout also raises the ceiling for the fast in-memory cells, weakening them as a guard; a live-cells-only budget keeps them strict at the cost of being carried explicitly. ⇒ Decide and say which, in the PR.
    2. What number, derived from what. ⛔ Not copied from driver-mongodb by analogy — 30_000 is that package's number, not a repo constant. Derive it from the observed live round-trip cost, the discipline [finding] sql-driver-11224-update-stamp-precision §3 now reds on a SLOW runner too — the span < 1000ms line is a validity precondition, and #11572's fix spends deliberate clock time inside that same budget #13691 applied to §3's span ceiling: a bound derived from the conclusion it guards, not picked. An undated, underived timeout is indistinguishable from one raised to silence a real regression.

    Sibling note: #16497 (core's discriminator pin, same default 5000ms) is graded p3 today — same class, but observed only on a shared dev box and never on CI. This card outranks it because the cost here is on the record.

    ⛔ This seat grades and routes only: not claimed, not dispatched, no code.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    domain:engineRetired 2026-10-22, alias of domain:server (engine surface); in-flight cards keep it until they landpriority:p2Medium: important, M3teststoolingFix lands in tooling, not a product package — enters pm:queue only with Unblocks: or a named surface

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions