Repository navigation
mcpx: the four genuine conformance gaps left at 2026-07-28, and the ~50 that are fixture artefacts not defects #1948
Description
Relocated from dezren39/darwin-nix#252 on 2026-10-28 (placement audit): mcpx's single source lives in developing-today/code (pkgs/mcpx); re-home tracked at #309. Original body below; issue references inside it are historical (dezren39/nix numbering).
Baseline from the official suite (@modelcontextprotocol/conformance@0.2.0-alpha.11) against the daemon's /mcp endpoint at 408bc2b:
| requirement set | passed | failed |
|---|---|---|
2026-07-28 |
115 | 54 |
2025-11-25 |
45 | 21 (2 of them #251) |
Read the 54 correctly before acting on it. About 50 are scenarios that never reach a protocol assertion, because they assume the suite's own fixture surface: tools named slow_compute, confirm_delete, failing_job, test_missing_capability, json_schema_2020_12_tool; prompts named test_simple_prompt, test_prompt_with_arguments; resources under test://; and a tool carrying x-mcp-header annotations. mcpx is a proxy — it publishes its own small tool set and namespaces upstream resources as mcpx://<namespace>/<uri> — so those report failures that say nothing about conformance. Every input-required-result-*, every tasks-*, prompts-get-*, tools-call-{image,audio,embedded-resource,mixed-content,with-progress}, http-custom-header-server-validation and json-schema-2020-12 failure is in this bucket.
Chasing that number as if it were 54 defects would be weeks of work for four fixes.
The four genuine gaps
1. -32021 MissingRequiredClientCapability is never raised.
sep-2575-server-rejects-undeclared-capability. When a tool needs a client capability the request's _meta did not declare, 2026-07-28 requires -32021 (HTTP 400). mcpx runs the tool. The code exists as codeMissingCapability in internal/mcpserver/server.go but nothing in the tools/call path consults the declared capabilities before dispatching.
2. The tasks extension does not gate on the extension capability.
sep-2663-tasks-methods-non-declaring: tasks/get, tasks/update and tasks/cancel must answer -32021 to a client that did not declare io.modelcontextprotocol/tasks; mcpx answers -32602. The extension overview is explicit: "Never return a task to a client that did not declare support."
3. No server-directed task creation.
tasks-server-directed-creation-no-hint, sep-2663-result-type-task-on-create. In 2026 the server decides per request whether to return a CreateTaskResult (resultType: "task"); the per-request opt-in of 2025-11-25 is gone. mcpx only ever creates a task when asked. Also CreateTaskResult must carry ttlMs (renamed from v1 ttl).
4. Mcp-Method on a tools/call changes the result.
tasks-headers-tolerate-mcp-method-on-tools-call: "tool result content MUST be unaffected by the Mcp-Method header." The body is the source of truth; headers are mirrors for intermediaries to route on. Something in the header path is feeding dispatch.
(1)-(3) are properly exercisable only against a server that publishes the fixture tools, which argues for #133's scriptable mock MCP server rather than guessing.
Also worth recording
- An official suite exists and is governance-critical. SEP-2484 (Final) makes a merged conformance scenario a condition of a Standards Track SEP reaching
Final; SEP-1730 ties official SDK tiering to a percentage from it (Tier 1 = 100%). Itsrequirements/<revision>.yamlsets are frozen on purpose. - The npm
latesttag is a 0.1.x that predates 2026-07-28. Install@alphaor you will score against the wrong revision. - SEP-1627, the earlier "conformance testing" proposal, is closed and superseded by SEP-2484; its golden-trace design was explicitly not carried forward. Reading 1627 and stopping is how you conclude no suite exists.
pkgs/mcpx/scripts/conformance.shalready runs both legs, including mcpx as the client under test viainternal/conformance/officialclient. It needsCONFORMANCE_DIR; the install is the only step needing network, and nothing ingo testtouches it.- A per-revision record of what a server owes, including the obligations the prose implies but never states, is now at
pkgs/mcpx/docs/spec/server-obligations.md§7.
Suggested next step
Wire scripts/conformance.sh into CI as a non-blocking job with a recorded baseline and an expected-failures file (the suite supports --expected-failures), so the number moves only deliberately. That is the cheap way to stop this regressing again — #251 is exactly the kind of drift it would have caught the same day.
Acceptance criteria
- The ~50 fixture-artefact failures are documented as artefacts (suite surface assumptions named), so the count is never again read as 54 defects.
- The four genuine gaps are each tracked by a live issue (the 🎨 Palette: Add ARIA labels to icon buttons #253–fix(ci): pin jules-action to v1.0.0 — @v1 does not exist upstream (#258) #259 set or successors) — this issue links them and closes when they do.
- A conformance re-run on current main replaces the 408bc2b baseline numbers in the matrix doc.
Activity
developing-today commentedon Oct 28, 2026
CODE-873
dezren39 commentedon Oct 28, 2026
History carried over from dezren39/darwin-nix#252 (verbatim, oldest first):
@dezren39 · 2026-10-27T15:36:22Z
Intake review
Fleet monitor intake pass, 2026-10-27 (repo-fleet-monitor).
Related: the 2026-09-30 mcpx conformance set — parent analysis #252, siblings #253, #254, #255, #256, #257, #258, #259 (filed together from
docs/spec/conformance-matrix.mdin dezren39/nix). Treat the set as one body of work; do not re-file per-gap issues.Notes: This is the parent analysis for the whole conformance set — its real content is the read of the 54 failures (≈50 fixture artefacts, 4 genuine). It already carried legacy
bug/testinglabels; normalized to the taxonomy here. Priority normal as the set's map; the genuine-gap children carry their own priorities.Risks: Treating this as a fix-it issue — it is a tracker/analysis item; work lands in the children.
Suggested approach: keep open as the set tracker; re-baseline after the next conformance run.
dezren39 commentedon Oct 29, 2026
Cross-reference from triage task 2824: relocated issue #1928 (mcpx tasks extension, from dezren39/darwin-nix#209) was closed as a duplicate of the #1554 set, with this issue named as the live owner of the residual tasks-extension conformance specifics — gaps 2 (extension-capability gating of
tasks/get|update|cancel) and 3 (server-directed task creation) above. Full mapping: #1928 (comment)9 remaining items
dezren39 commentedon Oct 30, 2026
🤖 Agent note · picked up
dezren39 commentedon Oct 30, 2026
🤖 Agent note · done (already implemented)
handleTaskinpkgs/mcpx/internal/mcpserver/tasks.goreturnsmissingTasks(-32021 with requiredCapabilities naming io.modelcontextprotocol/tasks) for modern tasks/get|update|cancel from a non-declaring client, checked before the task id. Pinned byTestTasksPerEra/2026-07-28/tasks/methods-from-a-non-declaring-client-are-32021in requirements_test.go — run on the laptop this shift: PASS.internal/mcpclient/modern.gosets the per-request clientCapabilities meta tonarrowCapabilities(own, ClientCapabilitiesFrom(ctx))(never wider than mcpx's own), so an upstream that requires a capability the caller did not declare raises -32021 itself, and mcpserver relays that error whole (UpstreamError in capability.go). mcpclient package tests PASS on the laptop this shift. (docs/spec/official-suite.md still describes the old defect — worth a doc refresh when the conformance baseline is next re-run.)dezren39 commentedon Nov 3, 2026
agent-note:v1
state: triaged → closed (all acceptance met at current main)
Agent: queue worker (repo-worker-code), OpenChamber scheduled lane 02cafe76
Found via: queue claim (task 7808, triage)
Verdict: all four genuine gaps are fixed on main, the ~50 fixture failures are documented, and the baseline is now generated + recorded in CI. Nothing here needs routing; closing with evidence.
Evidence at
origin/main314fbbe (2026-11-03):internal/mcpclient/modern.gonarrows per-requestclientCapabilitiesto the caller's declared set, and the relay carries the upstream's-32021whole (UpstreamError). Verified on main 2026-10-30 by task 3247 (tests PASS).handleTaskreturnsmissingTasks(-32021) before the task id for moderntasks/get|update|cancelfrom a non-declaring client; pinned byTestTasksPerEra/2026-07-28/tasks/methods-from-a-non-declaring-client-are-32021.CreateTaskResultwithttlMs/pollIntervalMs/status/createdAt/lastUpdatedAt—internal/mcpserver/requirements_test.go:862(2026-07-28/tasks/server-directed-CreateTaskResult-is-flat), plusfast-call-is-answered-directlyand the no-extension cases; scenario mapping ininternal/conformance/coverage_stateful.go:11-13.Mcp-Methodmust not change atools/callresult. Header handling is a mirror check only:checkModernHeaders(internal/mcpserver/transport.go:287-320) rejects disagreement with the body (-32020) and dispatch reads the body, so the header cannot alter the result (internal/mcpserver/transport.go:314).Acceptance criteria:
docs/spec/official-suite.mdnames the suite's fixture surface (referenceeverything-server.ts, the tasks SDK fixtures,internal/testsupport/taskmcp) and says to read failure lists, not totals.docs/spec/conformance-matrix.mdis now generated (do-not-edit, regenerated byTestMatrix) and.github/workflows/mcpx-conformance.ymlruns the official suite non-blocking with per-leg recorded baselines (scripts/conformance-record.py), exactly the suggested next step.Closing as done. Reopen (or file fresh) if a 2026-07-28 scenario regresses; the CI record is the guard.
dezren39 commentedon Nov 3, 2026
agent-note:v1
state: triaged (coverage pass)
Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
Found via: coverage pass 2026-11-03