Skip to content
This repository was archived by the owner on Nov 4, 2026. It is now read-only.
This repository was archived by the owner on Nov 4, 2026. It is now read-only.

changeset-release.yml: drop the duplicate pnpm test, and stop concurrency discarding release runs (95 of 200 never executed) #5404

Closed

Description

@os-support-ai

Blocked-by: #5403

Ruled by the maintainer on 2026-08-20 (#5393, verbatim 「接受你的所有建议,现在开始处理。」). Item 3 of 5, and last on purpose — see the ordering note below. Both halves are in changeset-release.yml, so one card, one PR.

Half 1 — the duplicate test run

Run tests (pnpm test, unsharded) is 34 minutes of a ~35-minute job; everything else combined is ~90 seconds (install 7 s, build 35 s, publish 12 s). It re-tests a commit that is already on main — failing it cannot keep anything out, it only blocks publishing after the fact.

Half 2 — runs are being discarded, not queued

concurrency: ${{ github.workflow }}-${{ github.ref }} with cancel-in-progress unset. I originally described this as serialising with a ~35-minute wait. #5395 refuted the magnitude and found something worse: GitHub holds only one pending run per group and discards the rest — 95 of 200 release runs never executed at all. Median wait for those that did run is 5 min 08 s (22% of wall clock), not ~35 min.

So the defect is not latency. Release runs are being thrown away, on a workflow whose last step publishes to npm.

⛔ Do not fix this with cancel-in-progress: true. The final step is changeset publish; cancelling mid-publish is worse than any wait. The requirement is that no merge loses its release run — find the shape that achieves that (a queue that actually queues, or a design where a dropped run is provably harmless because the next run subsumes it). If the second is true, prove it rather than asserting it: show that a discarded run's changesets are still consumed by the following run.

⚠️ Half 1 makes half 2 much cheaper — a ~2-minute job contends far less than a 35-minute one — but ⛔ it does not make half 2 unnecessary. A dropped run is a dropped publish at any duration.

⚠️ Why this card is last, and what must be true before it lands

I originally proposed removing the release-side tests first, arguing the CI push lane still ran the whole suite as a backstop. That argument was false: #5402 shows Test (coverage) has failed 100% of the time since 2026-08-16.

So this card may only land once both are true, and the implementer must verify both on origin/main before opening the PR, not take this card's word for it:

  1. ci.yml Test (coverage) has failed 100% of the time since 2026-08-16 — one test times out under v8 coverage, and Codecov has received nothing since #5402 is fixed — the push lane's suite actually passes.
  2. Shard Test (coverage) 4 ways and stop the Codecov upload being skipped by its implicit success() #5403 has landed — the push lane is sharded and its upload no longer fails silently.

If either is not true when you pick this up, stop and report rather than proceeding — removing this test run while the push lane is dark would leave main with no working post-merge validation at all.

⚠️ Related context the implementer should read but not act on: the merge queue currently runs no CI (#4986 — a repository-settings matter, ⛔ no PR can fix it), and the release PR itself runs no CI (#5397). Neither blocks this card; both mean there is less validation upstream than the workflow's design assumes.

⛔ Constraints

  • ⛔ changeset:publish behaviour is not in scope and must not change. ⛔ No AI-executed release.
  • ⛔ Do not touch changeset:version — fix(release): carry QUICK_REFERENCE's Current Release block in changeset:version #5396 just wired the QUICK_REFERENCE sync into it, and a pin in sync-quick-reference-release.test.ts asserts this workflow's version: input. If your change trips that pin, that is the pin working; do not weaken it.
  • Report elapsed from the jobs' own started_at/completed_at.

Evidence and method: #5395. Ruling: #5393.

Activity

  1. os-support-ai commented on Aug 20, 2026

    @os-support-ai
    CollaboratorAuthor

    Unblocked — and my stated precondition for this card was wrong

    Both gating conditions in the body are now met, and the second one needs a correction rather than a tick.

    1. Shard Test (coverage) 4 ways and stop the Codecov upload being skipped by its implicit success() #5403 landed — PR ci: shard the coverage lane 4 ways and make a skipped Codecov upload loud #5423, merged 17:03:13Z. The push lane is sharded (39 min → ~12 min) and a skipped upload is now loud.
    2. ci.yml Test (coverage) has failed 100% of the time since 2026-08-16 — one test times out under v8 coverage, and Codecov has received nothing since #5402's fix did what it claimed. I amended this card to say its precondition was "now The suite is still red under v8 coverage on main after #5402 — shard 3/4 fails deterministically and the unsharded run leaves no coverage/ at all #5421, not merely Shard Test (coverage) 4 ways and stop the Codecov upload being skipped by its implicit success() #5403", on the strength of The suite is still red under v8 coverage on main after #5402 — shard 3/4 fails deterministically and the unsharded run leaves no coverage/ at all #5421's report that the suite was still red under v8 coverage. The suite is still red under v8 coverage on main after #5402 — shard 3/4 fails deterministically and the unsharded run leaves no coverage/ at all #5421 has been refuted and closed not planned — its shard-3 failure was ci-cd-pipeline-doc.test.ts reacting to a workflow file that exists only on the throwaway measuring branch, whose sha1 happened to hash into shard 3/4. The instrument perturbed its own measurement.

    Current main: all four coverage shards green — CI run 32397046084 on 32ef595f4, plus a local --shard=3/4 --coverage at 431 files / 5546 tests / exit 0.

    So the backstop this card depends on — "the push lane actually runs and passes the suite" — is real. Label swapped pm:blocked → pm:dispatched.

    ⚠️ One live caveat, and it does not block this card

    Test (coverage) is still red on every push to main, but for a reason that has nothing to do with the suite: CODECOV_TOKEN is unset (Token length: 0; Codecov answers Token required - not valid tokenless upload). Filed as #5436; it needs a maintainer action plus a policy choice.

    That matters here only as a boundary: this card removes the release job's duplicate pnpm test because the push lane tests the same commit, and it does — the merge step succeeds and the thresholds are enforced there. The red is the upload, downstream of enforcement. ⛔ Do not treat #5436 as a reason to widen this card, and ⛔ do not "fix" the red by touching the upload — that is #5436's, and the loud-failure behaviour was accepted deliberately on #5403.


    Generated by Claude Code

  2. os-support-ai commented on Aug 20, 2026

    @os-support-ai
    CollaboratorAuthor

    Claim: PM loop round 22
    Session: session_01RV6yuVCxymHYE16PL9vQkE
    Branch: claude/issue-5404-release-lane-tests-and-concurrency
    Worktree: objectui-issue-5404
    Domain: repo:objectui
    File surface: .github/workflows/changeset-release.yml only (stop on breach; explain in the report)
    Container & model: M, mode:subagent, model: opus
    Clause-②: no — removes a duplicate test execution and changes how release runs are scheduled; no contract, no public surface, no assertion changed
    Serial constraints cleared: changeset-release.yml is untouched by everything in flight — #5365 owns packages/components + packages/layout, #5358 has no file surface, #5378+#5377 own plugin-detail/react/the guide. ⛔ .github/workflows/ci.yml belongs to #5403, merged, and is out of bounds here. ⚠️ changeset:version must not be touched: #5396 wired the QUICK_REFERENCE sync into it and sync-quick-reference-release.test.ts pins this workflow's version: input — if your change trips that pin, that is the pin working.

    Item 3 of 5 in the maintainer's 2026-08-20 ruling on #5393, and the last one outstanding.


    Generated by Claude Code

  3. added a commit that references this issue on Aug 20, 2026
    b808ce8
  4. claude commented on Aug 20, 2026

    @claude
    Contributor
    {
      "issue": 5404,
      "status": "done",
      "branch": "claude/issue-5404-release-lane-tests-and-concurrency",
      "pr": "https://github.com/objectstack-ai/objectui/pull/5443",
      "premise_still_valid": true,
      "summary": "Both halves landed in .github/workflows/changeset-release.yml, one PR, nothing else touched. Half 1: the duplicate `pnpm test` is removed — measured from each job's own started_at/completed_at over runs #3712-#3911, it was 31m19s of a 33m22s job (everything else combined ~90s), it re-tested a commit already on main, and it had already blocked 2 of the last 27 release-PR merges (#3606, #3901) by failing after the fact. Half 2: the concurrency group is now keyed by github.sha rather than by ref, so GitHub never holds a pending run to discard (93 of those same 200 runs were cancelled with an EMPTY jobs array — not one step ever ran), and serialisation moves into a fail-open `Wait for older release runs` step that orders by run_number and can only delay a release, never block one. I did not argue subsumption: the version half provably subsumes (changesets are files that accumulate on main), but the publish half does not — changesets/action@v1 publishes only when .changeset/ is empty, a state the next merged changeset destroys, so a discarded publish run is a permanently skipped npm version. That this is real and not theoretical: @object-ui/core's CHANGELOG declares 90 versions, npm has 74, and react/types are missing exactly the same 16.",
      "tests": "All at HEAD b808ce85f with a clean tree; heavy runs wrapped in flock -E 99 -w 540 /tmp/os-heavy-verify.lock. (1) Gate union, derived from the three-dot diff against merge-base 15236e055 (2 files: the workflow + one changeset): `npx vitest run --maxWorkers=2 scripts/__tests__` -> `Test Files 58 passed (58)` / `Tests 1571 passed (1571)`. scripts/__tests__ is the entire surface that reads .github/workflows/** and .changeset/** (grep-derived; this repo has no scripts/pm/dispatch-gates.mjs — that tool is objectstack's). (2) `pnpm check:control-bytes` -> OK (4495 tracked text files, 85 binary skipped). `pnpm changeset:check` -> fixed group OK + privatePackages declared + no major. `pnpm quick-reference:check` -> Current Release block already states every anchor. `node scripts/check-changeset-presence.mjs` -> no source of a released package changed, nothing owed. `node scripts/check-lint-coverage.mjs` -> 46/46 packages linted. (3) REVERSE VERIFICATION of the pin that guards this file, run from the COMMITTED state so restore was safe: changed `version: pnpm changeset:version` to `version: pnpm changeset version`. Expected direction red; observed red — sync-quick-reference-release.test.ts, `AssertionError: expected 'pnpm changeset version' to be 'pnpm changeset:version'`, `Tests 1 failed | 11 passed (12)`. Restored with `git checkout HEAD -- .github/workflows/changeset-release.yml` (tree clean afterwards) -> 12 passed. NO REBUILD APPLIES here and none was done: the pin reads the workflow file off the working tree, no dist/ is in the resolution path, so the build-the-mutation-leg rule has nothing to build — stated rather than silently skipped. (4) The wait loop cannot be exercised in CI (no AI-executed release), so it was run locally against RECORDED /actions/workflows/changeset-release.yml/runs payloads with a stand-in gh. Six cases, all exit 0: no older run -> proceeds at once; only NEWER runs in flight -> proceeds (a newer run must not block us); two older runs + cap reached -> polls, warns, proceeds; older run completes mid-wait -> proceeds at 6s, not at the 120s cap; gh api fails -> warns, proceeds (fail-open); this run is the oldest -> proceeds. (5) PRECONDITION re-confirmed on the exact base commit 15236e055 (ci.yml run 32402092139) rather than inherited from the card: all four `Test (coverage shard N/4)` green (609s/669s/648s/509s). `Test (coverage)` is red, and its step list shows `Merge the shard reports into one coverage report` SUCCEEDED with only `Upload coverage to Codecov` and the loud outcome step failing — enforcement works, the red is the upload (#5436), untouched. (6) YAML re-parsed after editing: concurrency group `${{ github.workflow }}-${{ github.sha }}`, cancel-in-progress false, permissions gained `actions: read`, 10 steps, no `Run tests`. Method note: every elapsed figure comes from the jobs API started_at/completed_at, never an assumed clock; the 93 never-executed runs are counter-probed by the same endpoint returning a populated jobs array for all 106 runs that did execute.",
      "open_questions": [
        {
          "question": "The 'no merge loses its release run' guarantee comes entirely from the per-sha concurrency group (3 lines of YAML). The `Wait for older release runs` step is a second, separable property — it preserves the serialisation the old shared group was trying to express — and it is ~25 lines of bash in a workflow that cannot be exercised in CI. Keep it, or ship the guarantee alone?",
          "options": [
            "A — what I shipped: per-sha group + fail-open FIFO wait. Nothing is ever discarded AND publishes stay serialised. The wait step is fail-open by construction (every exit path is exit 0, plus continue-on-error: true), so its worst case is exactly option B's behaviour: it can delay a release, never block one.",
            "B — per-sha group only, drop the wait step. Same guarantee, three lines, nothing to maintain. Cost: release runs can overlap. The realistic harm is not npm corruption (npm rejects a duplicate version atomically) but a stale release PR — two concurrent runVersion runs both force-push changeset-release/main and the later-finishing one wins, so an older commit's run can leave the release PR missing the newest changesets. That feeds #5442 (leftover changesets at the next release-merge commit = another skipped npm version), which is why I kept the ordering."
          ],
          "recommendation": "A, because the old `concurrency:` key expressed an intent — serialise this lane — that is correct; only its implementation (discard the pending runs) was wrong, and collapsing to B would quietly discard the intent along with the bug. The fail-open construction is what makes A safe to add to a lane I cannot execute: it degrades to B, never to a blocked publish. If a maintainer would rather carry no bash in this file, collapsing to B is a one-hunk removal of the first step and gives up no part of the card's stated requirement."
        }
      ],
      "out_of_scope_findings": [
        "filed as #5442: 16 of 90 versioned releases never reached npm (@object-ui/core's CHANGELOG declares 90, npm has 74; react and types are missing the identical 16, so these are whole releases, not per-package failures). The dominant cause is NOT concurrency and is not fixed by this PR: changesets/action@v1 publishes only when .changeset/ is empty, so a release-PR merge landing while newer changesets are pending takes runVersion instead, and the version it just bumped to is skipped forever. Load-bearing row: cfeb378b5 (run #3370, conclusion SUCCESS) carried 3 pending changesets, bumped core to 17.4.0, published nothing. Live state: repo at 17.6.0, npm dist-tags.latest 17.5.0. Filed unassigned and unlabelled, with the fix shapes left open because every one of them touches the publish path this card is forbidden to change."
      ]
    }

    Generated by Claude Code

  5. added a commit that references this issue on Aug 23, 2026
    478ec54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions