Skip to content

QA run · release-18.3.0-w4-mcp (5/5) · 7fa9873f · 2026-11-02 · 5 PASS / 0 PARTIAL / 0 FAIL / 0 BLOCKED / 0 NOT-RUN #28937

Open

Description

@objectstack-fleet

domain:devx#1 · os-warren · session_01JQwxbWBtYWqaV8NhvUrShB

Filed by the dispatching PM seat session for the W4-MCP runner session_01JQwxbWWuY7DoeEEbb5fxS6, whose own write was refused by its session permissions (RUNNER rule 2: staged text, posted verbatim through the same door).
Part of #27377 (wave 4, W4-MCP: the MCP live card, build + ask)

Selector release-18.3.0-w4-mcp: the five ai.* MCP items whose pins stop below the served door, driven live on real boots at the subject so every clause is scored at its own oracle layer (RUNNER rule 6). Part 2 of the card (18.3 MCP changes with no checklist item yet) is findings only and scores nothing; it is reported below and on the anchor. Every item has a verdict: 5 PASS / 0 PARTIAL / 0 FAIL / 0 BLOCKED / 0 NOT-RUN. No item is planned.

Environment fingerprint

  • Subject: framework 7fa9873f38f7a0158e3ce17338dcc0f446b113e3 (detached, never followed main). Console built in-repo from the same commit: pnpm build --concurrency=2 (117/117 tasks, 13m26s), then pnpm console:build --concurrency=2 (71/72 cached, copy only).
  • Node v22.23.4, pnpm 10.31.0, Playwright chromium from /opt/pw-browsers (consent-screen leg only).
  • App: examples/app-showcase. Every boot OS_PORT=PORT OS_STORAGE_LOCAL_ROOT=/tmp/w4mcp/RUN/uploads setsid pnpm -C examples/app-showcase exec objectstack dev --ui --seed-admin -p PORT -d file:/tmp/w4mcp/RUN/data.db (first boot compiled, later ones --no-compile). Sequential execution in one session; no subagents.
    • main 41520: MCP default-on, restarted twice on the same file DB (coded-refusals restart leg; the stdio leg borrowed the DB while it was down).
    • off 41521: OS_MCP_SERVER_ENABLED=false.
    • oauth (a) 41531 OS_AUTH_URL=http://10.0.0.5:41531 — address NOT held by the host (it holds only 127.0.0.1 and 192.0.2.2; no ip tool to add one); (b) 41532 OS_AUTH_URL=http://192.0.2.2:41532 (held, TEST-NET-1 = public to the rule); (b') 41533 = (b) plus OS_ALLOW_INSECURE_OAUTH_HTTP=true; (c) 41534 OS_AUTH_URL=http://localhost:41534 --log-level info; https 41535 OS_AUTH_URL=https://10.0.0.5:41535 --cert/--key (self-signed).
    • stdio: node packages/cli/bin/run.js serve -p 41541 --dev with OS_MCP_STDIO_ENABLED=true OS_MCP_STDIO_API_KEY=osk_… on the main DB, driven by the MCP SDK's stdio client.
    • Part 2 connector probes: a scratch copy of the showcase (removed after) on 41550/41551 with extra provider: 'mcp' instances against a local stub MCP server (127.0.0.1:47390: paged / endless / drift / hidden / 401 / 403 modes) and an oauth2 instance against the main boot.
  • Seed: --seed-admin (admin@objectos.ai), demo persona auditor.demo@example.com; osk_ keys via POST /api/v1/keys; OAuth tokens minted on the embedded AS (DCR application_type: native → authorize with the MCP resource → consent → token, PKCE S256).
  • Times: 2026-11-02 03:53Z to 04:14Z testing (wall), after 03:39Z–03:53Z environment.
  • Cleanup: scratch app removed; examples/app-showcase/.objectstack/installed-packages/ absent; git status clean.

Scope: selector and the revision each verdict is valid for

From W4-MCP-items.txt at 7fa9873: ai.mcp-http-surface r8 · ai.mcp-oauth-private-host-transport r6 · ai.skill-instructions-mcp-prompts r4 · ai.mcp-validate-expression r6 · ai.mcp-coded-refusals-and-runtime-exposure r4. Not re-run here (other cards): ai.mcp-stdio-fail-closed, api-backend.packaged-action-disabled-dispatch (W4-C1), ai.mcp-run-action-exposure-gate (W4-C3a), connector-mcp items (W4-C2b).

Per-clause verdicts

item · rev clause verdict layer · text evidence
ai.mcp-http-surface · r8 A0 pass HTTP-booted-server (off boot): anon GET /mcp/skill 404 RESOURCE_NOT_FOUND 'MCP server is not enabled'; osk_ key POST /mcp 404 same; session-cookie POST /mcp 404 same; anon POST /mcp 401 UNAUTHENTICATED 'Unauthorized: a valid API key is required' (switch not read). Observed the 404 (config-off), never the 501 (negative[1])
ai.mcp-http-surface · r8 A1 pass HTTP-booted-server: anon GET /api/v1/mcp/skill 200, content-type text/markdown; charset=utf-8, content-disposition inline; filename="SKILL.md", cache-control no-store; body lines 31/46 carry http://localhost:41520/api/v1/mcp
ai.mcp-http-surface · r8 A2 pass HTTP-booted-server: POST /mcp/skill 405, allow: GET, {success:false,error:{code:METHOD_NOT_ALLOWED,message:'Method not allowed — use GET',httpStatus:405}}
ai.mcp-http-surface · r8 A3 pass HTTP-booted-server: OAuth-live boot anon POST /mcp 401 'Unauthorized: a valid OAuth access token or API key is required' + WWW-Authenticate Bearer realm="ObjectStack MCP", resource_metadata="http://localhost:41520/.well-known/oauth-protected-resource"; off boot: 401 'Unauthorized: a valid API key is required', no WWW-Authenticate
ai.mcp-http-surface · r8 A4 pass HTTP-booted-server: admin osk_ key initialize 200 (protocol 2025-03-26, caps prompts+tools) then tools/list = exactly list_objects, describe_object, validate_expression, query_records, aggregate_records, get_record, create_record, update_record, delete_record, list_actions, run_action, resume_run (12, no degradation). Session bearer token: same 12
ai.mcp-http-surface · r8 A5 pass HTTP-booted-server: embedded-AS token scope 'openid profile' → 403 PERMISSION_DENIED 'Forbidden: the access token grants none of the MCP scopes (data:read, data:write, actions:execute, metadata:write)' + WWW-Authenticate error="insufficient_scope", scope="data:read data:write actions:execute metadata:write"; data:read token → the 6 read-family tools only; data:read+data:write+actions:execute → the 12; metadata:write gate both sides: key and session served no metadata:write tool, a metadata:write grant served list_remote_tables + import_object (see anchor accuracy line)
ai.mcp-oauth-private-host-transport · r6 A0 pass HTTP-booted-server, boot (a) on UNHELD 10.0.0.5 (fixtures.requires[0] rev 6: unheld stages the clause whole): both PRM paths 200 {resource:"http://10.0.0.5:41531/api/v1/mcp", authorization_servers:["http://10.0.0.5:41531/api/v1/auth"], 4 MCP scopes + offline_access}; anon /mcp 401 resource_metadata="http://10.0.0.5:41531/.well-known/oauth-protected-resource" — the advertised origin is the private one, never localhost (negative[1])
ai.mcp-oauth-private-host-transport · r6 A1 pass HTTP-booted-server, boot (b) 192.0.2.2: both PRM paths 404 ENDPOINT_NOT_FOUND, anon /mcp 401 with no WWW-Authenticate; log 1× 'OAuth discovery is served over PUBLIC plain HTTP' + 1× 'MCP server is enabled but the OAuth track is NOT live'; 0× 'OAuth is served UNENCRYPTED'
ai.mcp-oauth-private-host-transport · r6 A2 pass log: 'OAuth is served UNENCRYPTED' — (a) 1 at WARN naming http://10.0.0.5:41531/api/v1/auth; (c) 1 at INFO naming http://localhost:41534/api/v1/auth in its --log-level info log; default warn-level loopback boot (main) 0; (b) 0; https 0. 'OAuth discovery is served over PUBLIC plain HTTP' — (b) 1 naming issuer http://192.0.2.2:41532/api/v1/auth; (a) 0; (c) 0; https 0
ai.mcp-oauth-private-host-transport · r6 A3 pass HTTP-booted-server, boot (c): both PRM paths 200 with the (a) shape, origin http://localhost:41534 substituted; 401 carries the pointer
ai.mcp-oauth-private-host-transport · r6 A4 pass HTTP-booted-server, boot (b') with OS_ALLOW_INSECURE_OAUTH_HTTP=true: identical 404/404, 401 without pointer, same two log lines; git grep -l OS_ALLOW_INSECURE_OAUTH_HTTP → ai.json, plugin-auth CHANGELOG.md, auth-manager.mcp-oauth.test.ts, mcp-oauth-plaintext-notice.test.ts (no reader)
ai.skill-instructions-mcp-prompts · r4 A0 pass HTTP-booted-server: after PUT /meta/skill qa_prompt_full / qa_prompt_bare / qa_prompt_off (each 200), osk_ initialize declares prompts; prompts/list = [{name:qa_prompt_full,title:'QA Prompt Full',description:'Full QA skill'}]
ai.skill-instructions-mcp-prompts · r4 A1 pass HTTP-booted-server: prompts/get qa_prompt_full → message text 'QA-W4-INSTR-FULL: answer in one line.'; after re-PUT, the next get returns 'QA-W4-INSTR-FULL-V2' (re-read at get time)
ai.skill-instructions-mcp-prompts · r4 A2 pass HTTP-booted-server: list omitted qa_prompt_bare (no instructions) and qa_prompt_off (active:false); after PUT active:true (GET /meta/skill/qa_prompt_off reads active:true) the list carries both and get serves 'QA-W4-INSTR-OFF'
ai.skill-instructions-mcp-prompts · r4 A3 pass HTTP-booted-server: prompts/get qa_prompt_bare, qa_prompt_off (inactive, before activation and again after re-deactivation) and no_such_skill → JSON-RPC error -32602 'Prompt "…" not found' (negative[1] driven live)
ai.skill-instructions-mcp-prompts · r4 A4 pass plugin-unit [test oracle per verify]: packages/mcp skill-prompts.test.ts 12 passed at 7fa9873 incl. 'declares NO prompts capability when the host cannot read skill metadata'
ai.mcp-validate-expression · r6 A0 pass HTTP-booted-server: showcase_task 'record.estimate_hours * 1.1' formula → ok:true, errors [], warnings [], inferredType 'number', inScope {dialect cel, 7 roots, 26 fields incl estimate_hours, 38 functions}. Build parity: the same expression as a scratch formula field passes objectstack validate
ai.mcp-validate-expression · r6 A1 pass HTTP-booted-server: 'estimate_hours * 1.1' formula → error bare-reference 'Write record.estimate_hours'; 'record.estimate_hourz * 1' → error unknown-field '— did you mean estimate_hours?'; 'record.title * 2' → ok:true + warning type-mismatch. objectstack validate on the same three as formula fields: bare and typo refused with the identical sentences (rule expression-invalid), text-math not refused (negative[0] held)
ai.mcp-validate-expression · r6 A2 pass HTTP-booted-server: the SAME 'estimate_hours > 8' — site flow_condition ok:true errors [] (no inferredType); site formula ok:false bare-reference
ai.mcp-validate-expression · r6 A3 pass HTTP-booted-server: no_such_obj → isError 'Object "no_such_obj" not found'; sys_user → isError 'Object "sys_user" is a system object and is not exposed via MCP', no schema in the response. negative[1]: '' and ' ' at formula, '' at template → ok:false, one invalid_union error each (evaluated-slot sentence / template sentence), no inferredType
ai.mcp-coded-refusals-and-runtime-exposure · r4 A0 pass plugin-unit/engine: all 5 named files ≥1 test, 0 failed at 7fa9873: plugin.record-resource-exposure 8, mcp-action-refusal-details 5, mcp-coded-refusal 43 (packages/mcp), plugin-runtime-object-metadata-copy 10 (packages/objectql), mcp-stdio-runtime-object-exposure 10 (packages/qa/dogfood)
ai.mcp-coded-refusals-and-runtime-exposure · r4 A1 pass HTTP-booted-server: get_record {objectName:showcase_task, recordId:does-not-exist-w4} → isError {error:{code:RECORD_NOT_FOUND,message:'Record does-not-exist-w4 not found in showcase_task',status:404}}, same after restart; an id argument → -32602 naming id → recordId
ai.mcp-coded-refusals-and-runtime-exposure · r4 A2 pass HTTP-booted-server across a restart (same file DB): runtime qa_hidden_obj (enable.apiEnabled:false, sharingModel set) query_records → OBJECT_API_DISABLED 404 before and after; qa_shown_obj served both times; after DELETE /meta/object/qa_shown_obj (200) it left list_objects and describe_object answers 'Object "qa_shown_obj" not found'
ai.mcp-coded-refusals-and-runtime-exposure · r4 A3 pass HTTP-booted-server: staged qa_ai_mark (declarative update, ai.exposed) run_action as auditor.demo osk_ → {error:{code:PERMISSION_DENIED,message:'You do not have permission to perform this action. …',status:403}}, keys error.code/message/status only; admin control → ok:true, record updated

Derived: mcp-http-surface PASS · mcp-oauth-private-host-transport PASS · skill-instructions-mcp-prompts PASS · mcp-validate-expression PASS · mcp-coded-refusals-and-runtime-exposure PASS

Deltas against wave 3 (#27409, c4c515a)

  • ai.mcp-oauth-private-host-transport r5 → r6: PARTIAL → PASS. A0 blocked(fixture) → pass and A2 partial → pass: rev 6 states that an unheld private base URL stages A0 and A2's private leg whole, and this run met that shape. The only leg still unreachable here is steps[3] (a real MCP client on a second machine), which no clause requires.
  • ai.mcp-http-surface r7 → r8, ai.mcp-validate-expression r5 → r6, ai.mcp-coded-refusals-and-runtime-exposure r3 → r4: PASS → PASS.
  • ai.skill-instructions-mcp-prompts r4 → r4: PASS → PASS.

Reproduction rules

None: 0 clause fail. The product defects found beside the clauses and in Part 2 are extracted as their own cards, each with its own reproduction; the close-out comment lists them.

Fixture gaps

  • ai.mcp-oauth-private-host-transport steps[3]: no second machine and no held RFC 1918 / link-local address in this container (it holds 127.0.0.1 and 192.0.2.2 only).

Off-clause probes (RUNNER rule 8)

About 6 minutes after the clauses had verdicts, on the same boots and personas: prompts/get with an empty name (-32602, fine); a skill whose instructions are whitespace only (not projected, fine); re-deactivating a projected skill (withdrawn at once, fine); run_action on an unknown action name (refused, writes no audit row, fine); the id → recordId mistake on get_record (prescriptive -32602, fine); list_objects with an apiEnabled:false object (listed, as GET /api/v1/meta/object lists it; query refused, consistent, not filed). No defect on these doors.

Part 2 — 18.3 MCP changes with no checklist item (findings only, no verdicts)

build

ask

connector-mcp

Time: environment prep vs testing

  • Environment: 03:39Z–03:53Z (14 min wall): install 28 s, pnpm build 13m26s, console:build 1 s; probe scripts and the stub server written while the build ran.
  • Testing: 03:53Z–04:14Z (21 min wall): Part 1 ~11 min across 7 boots, Part 2 ~10 min across 11 boots (connector boot-fail cases ~12 s each).
  • Ratio about 2 to 3. Session cost at report time about $4.

Activity

  1. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    domain:devx#1 · os-warren · session_01JQwxbWBtYWqaV8NhvUrShB

    Filed by the dispatching PM seat session for the W4-MCP runner session_01JQwxbWWuY7DoeEEbb5fxS6, whose own write was refused by its session permissions (RUNNER rule 2: staged text, posted verbatim through the same door).

    Close-out for this record. 0 clause FAIL, so no FAIL dispositions are owed. Product defects found beside the clauses and in Part 2, each extracted after the pm-dispatch filing gate (the consent-icon note in Part 2 did not pass it and is not filed):

    Extracted: #28938 · ai.mcp-http-surface · observed beside the clauses
    Extracted: #28939 · ai.mcp-http-surface · observed beside the clauses
    Extracted: #28940 · ai.skill-instructions-mcp-prompts · observed beside the clauses
    Parked-on: #27377 · ai.mcp-oauth-private-host-transport steps[3] real-client leg (no second machine / held private address)
    Parked-on: #27377 · Part 2 items to author (build, ask, connector-mcp) and the accuracy lines below the anchor comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions