Repository navigation
[finding] service-automation: the durable run-history row folds cancelled (and timed_out) into failed — after a restart the Runs surface cannot tell a cancelled run from a failed one #15223
Description
Observed while delivering the services half of #13937 (branch claude/issue-13937-stranded-run-operator-verb); recorded, not fixed.
The reading, on origin/main 50d6c92
RunRecord.status(packages/services/service-automation/src/engine.ts) is declared'completed' | 'failed'.AutomationEngine.recordLogwrites the durable row for every terminal entry (completed,failed,cancelled,timed_out) withstatus: entry.status === 'completed' ? 'completed' : 'failed'.ObjectStoreSuspendedRunStore.deserializeTerminalreads it back asrow.status === 'failed' ? 'failed' : 'completed'; the in-memory store round-trips what it was given (failed).getRunprefers the in-memory ring entry, which still sayscancelled; after a restart (or ring-buffer eviction) the durable row answers, and it saysfailed.
⇒ In-process a cancelled run reads cancelled; after a restart the same run reads failed on getRun, listRuns and the Runs surface. restoreConsumedSuspension's refusal ladder can name RUN_CANCELLED only for a run this process saw cancelled — the durable row gives another replica nothing to distinguish (measured in stranded-run-status.test.ts on the named branch: the stale-copy refusal after a cancellation elsewhere is NO_CONSUMED_SUSPENSION, honestly, because that is all the row says). The narrowing is not explained in either file; ExecutionStatus in @objectstack/spec declares all four terminal members.
Not measured: whether any consumer branches on a durable cancelled today (plugin-approvals' dead-run sweep treats every terminal status alike, so it is unaffected).
Related
#13937 · #13909 · #2585 (run-history retention, the row's origin) · #7533 (the trigger columns on the same row)
Activity
os-zhuang commentedon Sep 4, 2026
分诊路由(本评论来自分诊座位)· R+150 ·
date -u实测 2026-09-04T20:09:35Z 一轮domain:services·finding·priority:p2。落点packages/services/service-automation/src/engine.ts(及两个 store 的读回路径)⇒ 车道表 services。p2 判据 —— 关键在于这是「写入即丢失」,不是显示问题:
recordLog在写持久行的那一刻就把cancelled/timed_out折成failed(entry.status === 'completed' ? 'completed' : 'failed')⇒ 信息不是被藏起来了,是不存在了。进程内的环形缓冲还记得,重启或淘汰之后永远回不来。⇒ 与「某个界面没显示」不同,这一类的代价随时间单调增加。⛔ 不是 p1:卡面自陈未测到任何消费者今天按持久
cancelled分支(plugin-approvals 的死 run 清扫对所有终态一视同仁,不受影响)⇒ 无已知的下游损坏。⭐ 提级条件:任何一个消费者开始按持久终态分支,立刻重估 —— 那一刻起,历史行已经是错的且不可修。⭐ 卡面里最有说服力的一处实测,派发时请一并交给实现者:
restoreConsumedSuspension的拒绝阶梯只能对本进程见过取消的 run 报RUN_CANCELLED;另一个副本读持久行什么也分辨不出(该分支上stranded-run-status.test.ts实测到的是NO_CONSUMED_SUSPENSION,而那是诚实的——因为行里只有这些)。⇒ 这条把「显示不准」升级成「另一个副本的操作动词答不出正确的拒绝理由」。RunRecord.status只声明两个成员,而@objectstack/spec的ExecutionStatus声明了四个 —— 两个文件里都没有解释这个窄化。⇒ 修之前先确认那个窄化是刻意的(有存储代价?)还是沿袭下来的;若是刻意的,把理由写在声明旁边,⛔ 别让下一个人再查一遍。Generated by Claude Code
os-zhuang commentedon Sep 8, 2026
分诊:补状态
pm:queue—— 唯一缺的就是它标签
findingdomain:servicespriority:p2维持不动 · 类型 Bug(声明的ExecutionStatus有四个终态成员,持久行只存两个 ⇒ 违背已声明契约)· 补pm:queueR+150 的分诊已定车道与等级,唯独没写状态标签。⇒ 落在 SKILL.md:325 的「析取 ③」里;派发 backlog 按
label:pm:queue取卡(:58) ⇒ 本卡此刻不在任何清单上。落点具名、四处读写链路已测清、无可问之事 ⇒ SKILL.md:341 入队。⭐ 本卡最锋利的一点:同一条 run,读两次得到两个答案
链路已被逐段测清,⛔ 不必重推:
RunRecord.status(engine.ts)'completed' | 'failed'—— 只有两个成员AutomationEngine.recordLogstatus: entry.status === 'completed' ? 'completed' : 'failed'ObjectStoreSuspendedRunStore.deserializeTerminalrow.status === 'failed' ? 'failed' : 'completed'getRuncancelled⇒ 进程内读
cancelled;重启后(或环形缓冲逐出后)同一条 run 在getRun/listRuns/ Runs 界面上读failed。⭐ 而
ExecutionStatus(@objectstack/spec)声明了全部四个终态成员。⇒ 这是「声明四个、持久两个」,且两个文件里都没有解释这次收窄 —— 一个读者无从知道这是刻意的还是遗漏。⭐ 它已经在使一条拒绝阶梯变哑
restoreConsumedSuspension的拒绝阶梯只能对本进程见过取消的 run 说出RUN_CANCELLED;换一个副本,持久行给不出任何可区分的东西。stranded-run-status.test.ts上实测:异地取消后的陈旧副本拒绝是NO_CONSUMED_SUSPENSION—— 而卡面用了 honestly 一词,是对的:那条拒绝没有说谎,它说的正是那一行所能支持的全部。⇒ ⛔ 缺陷不在拒绝阶梯,在它读的那一行。⛔ 卡面自己标注的 NOT measured,原样保留
⇒ 这条决定了代价的大小,却不决定它是不是缺陷 —— 持久层丢掉一个已声明的区分,本身就是缺陷。⚠️ 但它决定了修法的紧迫性:若确有消费者要分支,这就从「界面显示错误」变成「行为分叉」。
重判触发条件(写死):若发现任何消费者(平台内或应用侧)依赖持久的
cancelled/timed_out做分支,升 p1;若仅止于 Runs 界面显示,维持 p2。⛔ 认领时先做这一次普查,它决定修法范围(只加列 vs 加列并回填)。failed、还是标记为「不可判定」),⛔ 不要默认它们会自己变好。Related:#13937 · #13909 · #2585(run-history 保留,该行的出处)· #7533(同一行上的 trigger 列)· #15222(同一次交付记下的姊妹卡,同批补
pm:queue)⛔ 分诊席边界:不认领、不派发、不写码、不合并、不裁决。
Generated by Claude Code
yinlianghui commentedon Sep 9, 2026
Contract review (clause ②) — PASS · PR #17008 head
0ddabd202Reviewed in-seat at
claude-fable-5-1(CONTRACT_REVIEW_TIER) by the consolidated seatsession_018rzQyhLGC5iVs11V3TzRs5, 2026-09-09T02:36Z, under the maintainer's forced-consolidation instruction (given in this PM session's chat at 2026-09-09T01:32Z: 「强制接管其它pm的所有任务」). Standing trial (#16905 / #16915): built atTIER_DEFAULT, reviewed at the contract tier — this comment is the review half.Read: the full diff (7 files, +289/−29, all 488 diff lines), the card, all five comments, and the dispatching seat's p1 regrade (comment 5594809472).
Clause-②
yes— confirmed from the diff, not copiedRunRecord.statusengine.ts—'completed' | 'failed'→TerminalRunStatus(4 members)TerminalRunStatusindex.ts— added to theexport type {}blocksys_automation_run.statussys-automation-run.object.ts—Field.selectgainscancelled/timed_outonlyWhen.status.$inwidened to fourThree copies of the vocabulary collapsed to one (
TERMINAL_RUN_STATUSES+isTerminalRunStatus): the write fold inrecordLog(=== 'completed' ? 'completed' : 'failed'→terminalStatus), the read fold indeserializeTerminal(now takesstatusas a parameter resolved once by the gate inloadTerminal/listHistory), andlistHistory's inline two-member filter. Verified by hunk. The wire contract (ExecutionLog.status: ExecutionStatus) already declared all four, so no authoring accept set moves — the implementation stops under-reporting a member the contract promised.No runtime import cycle introduced:
suspended-run-store.tsnow imports the valueisTerminalRunStatusfrom./engine.js;engine.tsonorigin/mainimports nothing fromsuspended-run-store(0 hits), so the edge is one-directional.The three gates this seat is responsible for
.changeset/persist-terminal-run-status-distinction.mdgrades@objectstack/service-automation: minor. A clause-②yesPR must grade at least one moved packageminor(maintainer ruling 2026-09-04, batch [WIP] Add query enhancements and advanced validation features #35 on finding(changeset): two independent contract reviews read the repo's own history to opposite bumps for "add an exported symbol to a published index" #15294) — satisfied.**BREAKING**banner — not owed, agreed: the published contract promised four members before this row existed. One residual recorded below (exhaustive-switchshape), not blocking.sys_automation_runis an engine-owned system data table with no authorable surface, every value already stored stays valid under the widened option set, nothing is renamed or removed, so there is no migration prescription. The changeset carries no**BREAKINGtoken and the registration gate is green in CI.Ruled boundaries held
restoreConsumedSuspension's ladder is byte-unchanged (triage reserved it); the stale-hot-replica residue is recorded instranded-run-status.test.ts's comment, not mistaken for a fix.refuseddeliberately absent from the option set (declared by A flow cannot REFUSE with per-record text: the only channel that interpolates is a screendescription, and a message-only screen still renders Submit and toasts "completed" #14945, produced by no engine path — ADR-0078 inert-value rule).packages/spec/src/**path touched — the fix(analytics): ask the object-level read grant before serving an inline dataset — one admission verdict on every driver #16860 single-writer fence did not arise.Evidence quality
New describe (3 cases) reads every assertion through a second store over the same rows — the only place the defect is visible. The two ablation legs have distinct red signatures (write leg:
expected 'failed' to be 'cancelled'; read leg:expected 'completed' to be 'cancelled', and the ladder answeringRUN_COMPLETED), which is what proves the two folds are independent.p1 regrade stands.
classifyStrandedRunState(plugin-approvals) branches on a persistentcancelledand reads it throughautomation.getRun→loadTerminal; with the row carryingcancelled, the foreign-replica misreport (a deliberately cancelled run gradedunrepairable) closes with this PR. The existing green test that pinned a history-row state the store could not produce (stranded-request-inspection.test.ts) is now honest.Check runs on
0ddabd202(read 02:33Z, latest-per-name)33 names: 29
completed: success+ 4completed: skipped, 0 failed, 0 in progress. Governed paths: none (docs/adr/**,.claude/**,skills/**,AGENTS.md,CLAUDE.mdabsent).content/docs/releases/: untouched.Release
Next stroke by this seat, in order: clear
needs:contract-reviewon this card and on PR #17008 (four-step label write, read back),node --use-env-proxy scripts/pm/check-clause2-carriers.mjs --pair 17008, flip the PR ready, armauto-merge. The card closes viaFixes #15223when the queue merges.验收备注 (recorded, not filed)
SuspendedRunStore.recordTerminalwith an exhaustiveswitchoverRunRecord.statusfails to compile after this — the ordinary shape of aminorunion widening in this repo; the dev measured zero out-of-package readers ofRunRecord/TerminalRunStatus. 承接者: the changeset's own text, which says to widen the switch.refusedis a declared-but-inertExecutionStatusmember — arefusedrun would write no history row at all. 承接者: A flow cannot REFUSE with per-record text: the only channel that interpolates is a screendescription, and a message-only screen still renders Submit and toasts "completed" #14945's author.restoreConsumedSuspension'scancelledarm reads ring-first; a replica holding its own stalefailedentry still answersNO_CONSUMED_SUSPENSION. Triage-reserved. 承接者: whoever next opens the ladder.AutomationEngine.getRunreturnsnullfor both "no such run" and a swallowed store failure (check:durability-log-level结构性看不见「读接缝把故障答成空值」这一类 —— #4825 / #5108 全家都在闸门盲区里 #5186 shape). Pre-existing. 承接者: none today.Generated by Claude Code
yinlianghui commentedon Sep 9, 2026
Release: PR #17008 (head
0ddabd202) — carriers cleared and armed by the consolidated seatsession_018rzQyhLGC5iVs11V3TzRs5, on the contract review PASS above (comment 5594919804).needs:contract-reviewremoved on this carddomain:services, finding, pm:dispatched, priority:p1· assigneeos-trumpunchangedneeds:contract-reviewremoved on PR #17008documentation, size/m, tests, toolingscripts/pm/check-clause2-carriers.mjs --pair 17008auto-mergeMERGE)Check runs on the head at 02:33Z: 33 names, 29 success + 4 skipped, 0 failed. The card closes via
Fixes #15223when the queue merges; landing verified on a re-fetchedorigin/mainat the next patrol, not on the PR branch. The p1 regrade (comment 5594809472) stays on the card.Generated by Claude Code
yinlianghui commentedon Sep 9, 2026
Landed — PR #17008 merged 2026-09-09T03:04:35Z into
mainat775e5ec4b6(fix(service-automation): persist the terminal run status distinction (cancelled / timed_out survive a restart) (#17008)), read on a re-fetchedorigin/mainat 03:22Z, not on the PR branch. GitHub closed the card onFixes #15223at 03:04:37Z; the consolidated seatsession_018rzQyhLGC5iVs11V3TzRs5now stripspm:dispatchedand the assignee in the same stroke so the closed card carries no state label (priority:p1,domain:services,findingstay — ownership, not state).The contract review (5594919804) and its 验收备注 stand as the record; nothing further is owed on this card.
Generated by Claude Code
docs(automation): correct three declaration-truth residues on the run…
docs(automation): three declaration-truth residues on the run-history…
fix(service-messaging): sys_notification_delivery reaps its terminal-…
feat(service-automation): the flow `end` node honours `outcome: 'refu…