Repository navigation
Scenario runner: --model / --author-model / --ask-model, with the served model recorded per turn #31160
Description
epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu
Filing-gate category: ② maintainer-directed task. Maintainer, verbatim (Claude Code session, 2026-11-07), after asking 「你的画廊测试我可以用 workbuddy 或者 codex 测试吗？有没有入口？」, 「你现在本地测试画廊，用的什么模型」 and 「我用 Claude code 云任务可以测试画廊吗？」, then answering the seat's proposals: 「前面的建议都同意，立卡」. Routing, verbatim: 「能交给车队开发的你不要自己开发」. Epic: objectstack-ai/cloud#3600. Live runs are paused by the maintainer (cloud#3600, 2026-11-07), so verify with the recorded driver and unit tests; no live run is owed.
Problem
Every recorded turn of the 2026-11-06/07 runs used the login's default model (claude-opus-5-5: 17 build turns, 246 persona turns), and nothing lets a run pin or compare models. ClaudeCodeDriver already accepts a model option and passes --model (driver.ts), but the CLI does not expose it.
Ask
--model <alias>for every turn, plus optional--author-model/--ask-modelto set the build and persona turns apart. Thread them to the driver.- The run record and the scorecard state the model actually served per turn. The stream's
meta.modelis already parsed; show any mismatch with the requested model. - Tests: the flag reaches the CLI argument list; the record carries the served model.
Reader: the domain:devx lane seat, from pm:queue.
Generated by Claude Code
Activity
objectstack-fleet commentedon Nov 7, 2026
triage#1 · os-zhuang · session_01AavokzJ5DndAwitDXvKy4U
Path: none
Triage: audit · p1 kept ·
domain:devx· 2026-11-07T15:40Z⛔ Not a claim.
domain:devx#3's ask. It is maintainer-directed (the same order). The card adds--model/--author-model/--ask-modeltorun-scenarios(the driver already acceptsmodel) and records the model actually served for each turn, showing any mismatch.domain:devx,area:devpathandpm:queuekept. It is a small, self-contained change to the runner's CLI.Generated by Claude Code
objectstack-fleet commentedon Nov 7, 2026
domain:devx#3 · os-project-manager · session_01FEkQ5njkh3xbcDN6EtSfw7
Claim: PM loop round 1
Session:
session_01FEkQ5njkh3xbcDN6EtSfw7Account:
os-project-managerBranch:
claude/project-thread-daow8a(a project card thread,session_01FEkQ5mMXSZe9L7NVZxrmUk; the thread can push only its harness branch, a measured platform exception toclaude/issue-<n>-<slug>)Worktree:
objectstack-issue-31160Domain:
domain:devxSeat:
domain:devx#3File surface:
packages/qa/scenarios/src/runner/**(cli.tsflags,driver.tsmodel threading,transcript.ts/scorecard.ts/ run record served model per turn), its tests and the package README's flag list.Container & model:
S,mode:thread,model: default — dispatch-gates --tier: no path-derived mandate,effort: mediumClause-②: no
Metadata-protocol: none
Responsibility: n/a — not a defect card; maintainer-directed runner option
Thread-read: 6709183409
Premise: holds.
run-scenarioson main56baf78etakes no--modelflag; the card says the driver already acceptsmodel.Serial constraints cleared: PR #31188 (#31147) edits
runner/cli.tsandrunner/scorecard.tsand is in review; the order has the dev mergeorigin/mainafter it lands before editing those two files. #31159 (same files) is queued behind this card.Order:
pm-orders/devx/order-31160.md(project files), handed to the card thread in its opening message.objectstack-fleet commentedon Nov 7, 2026
domain:devx#3 · os-project-manager · session_01FEkQ5mMXSZe9L7NVZxrmUk
os-dev-report
{
"issue": 31160,
"status": "done",
"branch": "claude/project-thread-daow8a",
"pr": "#31204",
"session": "session_01FEkQ5mMXSZe9L7NVZxrmUk",
"premise_still_valid": true,
"summary": "run-scenarios gains --model (every live turn), --author-model (build and iterate turns) and --ask-model (question and operation turns), threaded through makeDriver to ClaudeCodeDriver, which passes
claude --modelper kind of turn; a replay refuses all three like the live-driver limits, and an empty or flag-shaped value is refused. Each turn records meta.process.requestedModel beside the stream's meta.model; TurnSummary carries model, requestedModel and modelMismatch (servedAsRequested: the served name must hold the requested one as whole dash-separated parts). The scorecard's author rows name the served model (with a mismatch mark), newRun · model/Iterate · Run · modelrows count persona turns per served model, and HalfScores.models carries the tally into the record; README lists the flags. PR #31188 (same cli.ts / scorecard.ts) was still open: origin/main was merged once at 8acbfab before the PR (head 0b43e74, no conflict); per the order this branch merges origin/main again after #31188 lands, and its RunRecordMeta.packages (required there) will need the new orchestrate.test.ts runRecord call to passpackages— I will do that merge when told it landed. The PR body is written once (dev rule); the post-merge Verification refresh is the seat's.","tests": "On head 0b43e74 after
pnpm --filter \"@objectstack/scenarios...\" buildunder the verify lock (VERDICT command-exit 0):pnpm --filter @objectstack/scenarios test-> Test Files 23 passed (23), Tests 460 passed (460);pnpm --filter @objectstack/scenarios typecheckexit 0 (tsc --noEmit, --listFiles includes test/runner/units.test.ts). New: units.test.ts 3 cases (servedAsRequested 12-row table; --model / --author-model / --ask-model reach the stand-in claude argv with requestedModel and served meta.model per turn; refusal of missing/empty/blank/flag-shaped values and replay refusal of each model flag), orchestrate.test.ts 2 cases (crm card through runCard with a RecordingDriver: per-turn model/requestedModel/modelMismatch in cardRun, first.models tally, rendered author row and Run · model rows with the mismatch mark, JSON round trip of runRecord with env.driver.authorModel/askModel, and the saved recording replayed through RecordedDriver.fromFile answering both names; a run naming no model keeps| Run · model | — |). Ablations via scripts/ablation-replace.mjs on 4798cbf, src imported directly (no dist leg), each restored with blob == HEAD and git diff HEAD empty: (1) mismatch marking disarmed in orchestrate.ts summarize -> orchestrate -t served 1 failed | 2 passed; (2) replay refusal of model flags disarmed (LIVE_FLAGS.slice(0, 3)) -> units -t model 1 failed | 6 passed; (3) empty/flag-shaped value refusal disarmed -> 1 failed | 6 passed; (4) --model push in driver.ts base disarmed -> 1 failed | 6 passed.","gates": "
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackon 0b43e74 (merge base 8acbfab, 8 paths): 49 commands, each run with its exit captured before any pipe: 48 exit 0;pnpm check:dual-build-cjs-loadsexit 3 PREREQUISITE NOT MET (needs every package dist; only the scenarios closure was built) = NOT MEASURED, left to CI.--ranreconciliation: 49 derived famil(ies) accounted for — 48 run, 1 NOT-MEASURED (1 DERIVED from a recorded exit 3). Scoped lintnode --stack-size=4000 node_modules/eslint/bin/eslint.js packages/qa/scenarios --no-inline-configexit 0 on 0b43e74. CI check-runs on PR #31204: in_progress at report time (the merge commit 0b43e74 was pushed right after pr_create, so the PR head moved 4798cbf -> 0b43e74).","dogfood_battery": "n/a — no
!in the PR title and no (narrowing) arm","mcp_calls": "5 — reads only: issue_read get #31160, issue_read get_comments #31160 x2 (card + claim check), pull_request_read get #31188, pull_request_read get #31204 (label/draft read-back); no MCP write",
"api_writes": "2 — relay pr_create (fleet-write request fw-20261107T172907Z-a2f63f, run 42396532154, body read back identical 13922 bytes, labels skip-changeset, draft, no assignees) and this os-dev-report comment on #31160 via post-stamped.mjs; plus 2 git pushes (4798cbf, then the merge 0b43e74), not REST writes",
"open_questions": [],
"security_findings": "none",
"out_of_scope_findings": []
}
Generated by Claude Code
objectstack-fleet commentedon Nov 7, 2026
domain:devx#3 · os-project-manager · session_01FEkQ5njkh3xbcDN6EtSfw7
Landed 2026-11-07T19:45Z · PR #31204 merged through the merge queue at 2026-11-07T19:45Z, as
0d0c377beb988b990e77aa4763fd5e4b429d8de4.merged: true, merge commit0d0c377beb, head reviewed45ceeaff29(ACCEPT on the PR)completedbyFixes #31160State change:
pm:dispatchedremoved (card closed). Next: #31159 (the external driver contract) was serial behind this PR onrunner/cli.ts,driver.tsandorchestrate.ts; it is dispatched from this seat now.