Repository navigation
Scenario runner: an external driver contract (--driver external:<cmd>) and a Codex driver, so the same card can measure other agent tools; check WorkBuddy #31159
Description
epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu
Filing-gate category: ② maintainer-directed task. Maintainer, verbatim (Claude Code session, 2026-11-07), after asking 「你的画廊测试我可以用 workbuddy 或者 codex 测试吗？有没有入口？」, 「你现在本地测试画廊，用的什么模型」 and 「我用 Claude code 云任务可以测试画廊吗？」, then answering the seat's proposals: 「前面的建议都同意，立卡」. Routing, verbatim: 「能交给车队开发的你不要自己开发」. Epic: objectstack-ai/cloud#3600. Live runs are paused by the maintainer (cloud#3600, 2026-11-07), so verify with the recorded driver and unit tests; no live run is owed.
Problem
run-scenarios drives only the local Claude Code CLI (--driver claude) or a replay (--driver recorded:F) (packages/qa/scenarios/src/runner/cli.ts, makeDriver). So the same card cannot measure another agent tool (OpenAI Codex CLI, Tencent WorkBuddy, …). Yet a side-by-side comparison of agent tools on the same verified card is exactly what the gallery can show.
Ask
- An external driver contract. Document the
AgentDriverseam (driver.ts:author(task)builds in a project directory with shell and file tools;ask(task)answers one persona turn over the app's MCP with that persona's API key and returns aParsedTurn: reply text, tool calls and outputs, meta) as a process contract. An external command gets the task on stdin as JSON and prints the turn as JSON. Add--driver external:<command>. Keys go in a 0600 file, never on argv, as the Claude driver does. - A Codex driver (
--driver codex) on that contract, using the Codex CLI's non-interactive mode with JSON output and its MCP configuration foraskturns. Measure the real flags on the installed CLI; do not assume them. Refuse an API-key login the same way the Claude driver does, unless the maintainer rules otherwise. - WorkBuddy: check whether it has a headless mode with structured output and MCP client support. If it does, wire it through the external contract. If not, say so in the PR; do not build a UI bridge.
- The run record and the scorecard name the driver and its version (already done for Claude). The gallery exporter (Scenario library T8: export verified scenario apps to a new gallery repository — source, run record, auto screenshots and the 8–10 问数 transcripts, rebuilt per release #30349) shows the driver on each app page.
Reader: the domain:devx lane seat, from pm:queue.
Generated by Claude Code
Activity
objectstack-fleet commentedon Nov 7, 2026
triage#1 · os-zhuang · session_01AavokzJ5DndAwitDXvKy4U
Path: none
Triage: audit · p1 kept ·
domain:devx· 2026-11-07T15:39Z⛔ Not a claim.
domain:devx#3's ask (6709157326). It is maintainer-directed (「前面的建议都同意，立卡」, epic objectstack-ai/cloud#3600). The card asks for an external driver contract (--driver external:<cmd>, theAgentDriverseam as a stdin/stdout JSON process contract) and a Codex driver, so the same card can measure other agent tools; it also asks to check WorkBuddy.domain:devx,area:devpathandpm:queuekept. Live runs are paused, so the card is verified with the recorded driver and unit tests.Generated by Claude Code
objectstack-fleet commentedon Nov 7, 2026
domain:devx#3 · os-project-manager · session_01FEkQ5njkh3xbcDN6EtSfw7
Claim: PM loop round 1
Session:
session_01FEkQ5njkh3xbcDN6EtSfw7Account:
os-project-managerBranch:
claude/project-thread-mickba(a project card thread,session_01FEkQ5mx64pxVGCbKfuxEwi; the thread can push only its harness branch, a measured platform exception toclaude/issue-<n>-<slug>)Worktree:
objectstack-issue-31159Domain:
domain:devxSeat:
domain:devx#3File surface:
packages/qa/scenarios/**only:src/runner/{driver.ts,cli.ts}(or a newsrc/runner/drivers/file per driver),src/export/**for the driver line on each app page,README.mdfor the external contract, tests and a fake external command fixture undertest/**;orchestrate.tsonly if the driver seam needs it. Plus one changeset. Nothing else.Container & model:
L,mode:thread,model: default — dispatch-gates --tier: no path-derived mandate,effort: mediumClause-②: yes — after #31221,
@objectstack/scenariosis published; a new--drivervalue and any exported driver class or type widen its CLI and./runnersurface (additive). Contract review owed atTIER_DEFAULTon the head before landing (in-seat, per Leehom 「你有 fable ，车道契约复审可以自己做」 2026-11-07T16:00Z).Metadata-protocol: none
Responsibility: n/a — QA tooling, no product door
Thread-read: 6709176909
Premise: holds. On main
0d0c377be,makeDriverinpackages/qa/scenarios/src/runner/cli.tsstill refuses any driver but the two (line 188: "unknown driver … (claude, recorded:FILE)").Serial constraints cleared: PR #31204 (#31160;
cli.ts,driver.ts,orchestrate.ts) merged 2026-11-07T19:45Z as0d0c377be. PR #31221 (#31172;cli.ts,package.json,src/bin.ts) is in the merge queue: the thread starts only after it merges, and the seat tells it when. #30914 (sameorchestrate.ts) waits behind this card.Ruling-ref: maintainer-directed p1 (「前面的建议都同意，立卡」, epic objectstack-ai/cloud#3600); live runs paused (cloud#3600, 2026-11-07), so verification is unit tests and the recorded driver.
Order:
pm-orders/devx/order-31159.md(project files), handed to the card thread in its opening message.Generated by Claude Code
objectstack-fleet commentedon Nov 7, 2026
domain:devx#3 · os-project-manager · session_01FEkQ5mx64pxVGCbKfuxEwi
os-dev-report
{
"issue": 31159,
"status": "done",
"branch": "claude/project-thread-mickba",
"pr": "#31265",
"session": "session_01FEkQ5mx64pxVGCbKfuxEwi",
"premise_still_valid": true,
"summary": "run-scenarios gains --driver external:COMMAND (ExternalDriver, contract objectstack-scenario-driver/1: one process per call, the call as strict JSON on stdin for info/author/ask, the turn as strict JSON on stdout; a non-zero exit is a failed turn named no-result(exit N), another answer is invalid-output(why), never an empty pass; flags go in the stdin limits) and --driver codex (CodexDriver on codex exec --json, flags measured on @openai/codex 0.167.0: refuses OPENAI_API_KEY / CODEX_API_KEY in the environment and any login status other than a ChatGPT login, forces forced_login_method=chatgpt per turn, --model/--author-model/--ask-model map to -m, --author-max-turns/--ask-max-turns refused at makeDriver since codex exec has no turn cap). The persona MCP key reaches either tool only in a 0600 file (external: its path is in the ask stdin JSON; Codex: a temporary profile under CODEX_HOME read with -p), removed when the turn returns; never argv, environment or stdin value. DriverInfo gains replayOf so a recorded replay names the recording's driver and version, and each gallery app page carries a '- Driver:' line (driverText). README documents the contract and the WorkBuddy check (no headless mode with structured output and an MCP client found, so no wiring and no UI bridge). Live model runs paused per the order: verified with unit tests and stand-in commands only.",
"tests": "On head fb1d76f after the scenarios closure build:
pnpm --filter @objectstack/scenarios build && pnpm --filter @objectstack/scenarios testexit 0 -> Test Files 30 passed (30), Tests 573 passed (573);pnpm --filter @objectstack/scenarios typecheckexit 0. New: test/runner/external-driver.test.ts 8 cases against fixtures/fake-external-driver.mjs (info refusal on exit/prose/missing; author stdin shape, cwd, turn mapping, snapshot; ask 0600 config file, key on none of argv/env/stdin, file and dir removed; no-result(exit 3), invalid-output for prose and wrong shape, timeout kill; command parsing; limits on stdin and record; end to end: the crm_account_with_contacts card through runCard in both modes via the stand-in command with key sk-scenario-e2e-key-9f27c1d4, then the run record, recording, scorecard and full gallery export grepped: zero files hold the key; the app page reads '- Driver: external:fake-agent 1.2.3' and a replay names replayOf). test/runner/codex-driver.test.ts 12 cases against a stand-in codex (JSONL parsing on 0.167.0 shapes, cost guard, profile TOML, argv per turn kind, 190 KB prompt on stdin, profile 0600 and removed, no billing var in the child, user server named app refused, timeout/no-result/turn.failed not measured, model flags to -m, turn-cap flags refused). First run of both new files: Test Files 2 passed (2), Tests 20 passed (20). Ablations via scripts/ablation-replace.mjs on fb1d76f, src imported directly (no dist leg), each restored with blob == HEAD and git diff HEAD empty: (1) external.ts ask writes the key into the turn's meta.model -> external-driver.test.ts 1 failed | 7 skipped, the e2e grep naming 7 files (run record, recording.json, scorecard.md, gallery README.md, run-record.json, scorecard.json, scorecard.md); (2) codex.ts login guard widened to accept 'API key configured' -> codex-driver.test.ts -t 'cost guard|refuses an API-key login' 2 failed | 2 passed; restore leg both files 20 passed (20).","gates": "
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackon fb1d76f (BASE 169ff7e, 15 files): each command run with its exit captured before any pipe, all exit 0 except:node scripts/check-plugin-teardown-shape.mjs --self-testexit 3 on the shallow clone, re-run exit 0 after fetching its pinned commit 621a487;pnpm check:dual-build-cjs-loadsexit 3 PREREQUISITE NOT MET (needs every package dist; only the scenarios closure was built) = NOT MEASURED, left to CI (the package's own dist cjs and esm entries were loaded directly, OK).--ranreconciliation: 60 derived famil(ies) accounted for, 58 run, 2 NOT-MEASURED (both from a recorded exit 3; teardown since re-measured exit 0). Outside the derivation: rootpnpm check:error-code-provenanceexit 254 (no such root script),pnpm --filter @objectstack/spec run check:error-code-provenanceexit 0. Scoped lintnode --stack-size=4000 node_modules/eslint/bin/eslint.js packages/qa/scenarios --no-inline-configexit 0;node scripts/check-nul-bytes.mjsexit 0;GITHUB_EVENT_NAME=pull_request node scripts/check-changeset-no-major.mjs --base origin/mainwith the PR body exit 0. Tier: no path-derived mandate, 1608 changed lines (under 3000). CI check-runs on PR #31265 head fb1d76f at report time: Test UI success, filter success, the UI/docs/opt-in lanes skipped by path; Build Core, Plan Test Core shards, Type Check (workspace, source, consumer, debt ledger), Lint (ESLint, scoped tooling self-tests, repo and tooling gates, docs and code guards, CI release and engine gates, PM dispatch-gates), Dogfood Regression Gate and Live E2E UI in_progress.","dogfood_battery": "n/a — no
!in the PR title and no (narrowing) arm","mcp_calls": "0 — no GitHub MCP call; card, claim, #31221 and check-run reads went through gh api",
"api_writes": "2 — relay pr_create (fleet-write request fw-20261107T204349Z-99e871, run 42412445446 success, PR #31265 draft, body read back identical 22945 bytes, no labels, assignees none) and this os-dev-report comment on #31159 via post-stamped.mjs; plus 1 git push (fb1d76f), not a REST write",
"deviations": [
"os-dev rule 2 says pr_create carries assignees; the order says no assignees, and the order was followed (assignees read back: none).",
"pnpm check:dual-build-cjs-loads NOT MEASURED locally (exit 3, needs every package dist); CI runs it.",
"Ask 2 measurement: @openai/codex 0.167.0 and the npm WorkBuddy tarball were installed and read in a scratch prefix outside the repo; no codex login was used and no model turn ran (the proxy refuses OpenAI hosts with 403)."
],
"files_changed": 15,
"line_budget": "1496 insertions, 112 deletions across 15 files (git diff --stat origin/main...HEAD), all under packages/qa/scenarios plus one changeset",
"open_questions": [],
"security_findings": "none",
"out_of_scope_findings": []
}
Generated by Claude Code
objectstack-fleet commentedon Nov 7, 2026
domain:devx#3 · os-project-manager · session_01FEkQ5njkh3xbcDN6EtSfw7
Landed 2026-11-07T21:26Z · PR #31265 merged through the merge queue at 2026-11-07T21:26Z, as
d3b6ff431729a1079757a613dd52186d4418ed87.merged: true, merge commitd3b6ff4317, head reviewedfb1d76f7d2(contract review 6712878828 PASS, ACCEPT 6713022153)completedbyFixes #31159State change:
pm:dispatchedremoved (card closed). Seat ruling recorded in the ACCEPT: anexternal:command's billing variables are stripped, not refused. Not filed (follow-up if the maintainer wants it): READMEinfogrammar and timeout, a stale Codex profile after a SIGKILL, Codexaskread-only shell parity, the doubledcodex-cliname on the gallery line.