[finding] getMetaDiagnostics swallows the 503 that #5532 raised — an unreadable metadata store is published as "0 problems", and the type vanishes from the Studio tile counts

Filed by the domain:metadata PM seat off evidence surfaced during #6504 (list-shaped reads whose count is published as a whole-corpus fact). It is deliberately not folded into #6504 — see "Why this is its own card" — and it is filed ungraded: the type field and the domain:* routing are triage's to produce.

Everything below is read from origin/main. Nothing here has been executed; the reproduction is written out but not run, and that is stated where it matters.

What it publishes

packages/metadata-protocol/src/protocol.ts:4637 — getMetaDiagnostics() returns four numeric/aggregate facts about the metadata corpus:

total          // count of entries failing spec validation
scannedTypes   // = targetTypes.length
scannedItems   // incremented once per item actually walked
stats          // keyed by type name -> { count, locked, packages };  count = items.length

Its own doc names the consumer, so the audience is not inferred:

Per-type aggregate stats — count of items and the list of packages contributing to each type. Computed in the same sweep so the Studio directory page can render tile counts and a package filter in one round-trip.

The REST door is packages/rest/src/rest-server.ts:4759 — GET {metaPath}/diagnostics — which returns the object verbatim (res.json(result)). No shaping, no degradation flag.

The swallow

The per-type read is the only thing between that payload and the store:

for (const t of targetTypes) {
    let listed: any;
    try {
        listed = await this.getMetaItems({ type: t, organizationId: …, packageId: … } as any);
    } catch {
        // Type not listable in this kernel scope — skip.
        continue;
    }
    …
    stats[t] = { count: items.length, locked: lockedCount, packages: […pkgSet].sort() };
}

The comment names a benign reason, and that reason is real. The catch is untyped and takes everything else with it — including the one error the callee exists to raise.

getMetaItems is protocol.ts:5129. It calls rethrowUnlessMetadataStoreUnprovisioned at :5310 and :5369, and that guard (protocol.ts:4760) throws metadataStoreUnavailableError — a 503 — for every read failure that is not "sys_metadata has not been provisioned yet". Its doc states the rule this card is about, ~130 lines below the swallow, in the same file:

ADR-0110 D3 is the rule: a miss and an outage are different facts with opposite meanings, and a consumer must never read one as the other.

and lists, among the three defects #5532 removed:

getMetaItems turned it into items: [] — an outage answered as "this environment declares none of these"

#5532 raised that 503 out of getMetaItems precisely so an outage would stop looking like emptiness. getMetaDiagnostics catches it back into emptiness one layer up, and then publishes the emptiness as a number.

Why the worst value is the benign one

Three dispositions, none of which the caller can tell apart from a healthy answer:

Per type — an unreadable type produces no stats[t] key at all. Not zero: absent. The payload is byte-shaped like an environment that declares none of that type.
Corpus-wide — total counts entries that failed validation, and skipped types contribute none. A store that is entirely unreachable therefore answers total: 0 — a diagnostics endpoint whose whole job is to report problems reports zero problems when it cannot read anything. Green is the failure mode.
scannedItems under-counts silently, with no companion field saying by how much.

`scannedTypes` is the field that would have caught it, and it is computed from the intent

const targetTypes = request.type ? [request.type] : DEFAULT_METADATA_TYPE_REGISTRY.filter(…).map(…);
…
return { entries, total: entries.length, scannedTypes: targetTypes.length, scannedItems, stats };

targetTypes is fixed before the loop and never decremented on continue. So the response asserts "I scanned N types" while having scanned fewer, and the discrepancy is not derivable by the caller either — Object.keys(stats).length is only a lower bound, since a type that reads fine but holds no items also produces a stats entry (count: 0). There is no field in the payload from which a client can compute "some of this is missing".

What a fix must not do

⛔ Not "delete the catch". The benign reason in the comment is genuine — a kernel scope where a type is not listable must not 500 the whole diagnostics sweep. The discrimination already exists in this file and is the platform's single answer to this question: classify by error type through the same isMissingTableError / rethrowUnlessMetadataStoreUnprovisioned path that DatabaseLoader (#5108) and SysMetadataRepository (#4867) ask. Whether the outage case should then propagate the 503 or be surfaced as a per-type degradation marker in the payload is the actual design call on this card, and it is a read-surface change to metadata-protocol — which is why it is not consumer-sweep work.

Why this is its own card, not part of #6504

#6504 swept direct consumers of a list()-shaped read that publish its length as a corpus fact. getMetaDiagnostics is two hops away — it consumes getMetaItems, not list() — and its cleanest fix reads the verdict off the metadata service rather than hardening a call site. Folding it into #6504 would have widened that card after the fact instead of completing it, and would have pre-empted this routing. #6504's dev surfaced the evidence and correctly declined to file it from inside the card.

The relationship to #6504 is a shared shape, not a shared fix: a count taken from a read that can silently return less than the whole, published as though it were the whole.

Reproduction — written, NOT run

Stub an engine whose sys_metadata read throws a non-isMissingTableError driver error, then:

GET /api/v1/meta/diagnostics

Expected under the defect: 200 with total: 0, scannedItems: 0, stats: {}, and scannedTypes still reporting the full registry count (undecremented). Whoever takes this should measure it before writing anything — I have read the path, not executed it, and the direction should be predicted before the run.

Adjacent observation — UNVERIFIED, do not act on it from this card

The REST handler forwards type, severity and packageId but no organizationId, while getMetaDiagnostics accepts one and forwards it to getMetaItems. Whether an undefined organizationId widens this read across tenants was not measured, and this card asserts nothing about it. It is recorded here only so it is not lost; it belongs with the tenant-scoping thread (#8805, and the audit-read scoping that landed for #8747), and needs its own measurement before it is anybody's card.

Triage note (input, not a claim)

Fix lands in packages/metadata-protocol/src/protocol.ts (with packages/rest in the consumption radius if the payload gains a degradation marker) — reading as domain:metadata, but the domain:* label is triage's to produce and is deliberately left off. Under the mechanical boundary test this looks like Bug rather than Feature: the acceptance set does not widen, and the change restores a declared contract (ADR-0110 D3) that this file states in its own prose.

⚠️ Editorial note: this body was posted, read back, and re-posted once — the first write had two angle-bracket spans silently eaten by the sanitizer (a generic type in the field list, and a placeholder in the reproduction). Both are rewritten above without angle brackets. Nothing else changed.

Triage (finding first-touch): promoted — finding ⇒ pm:queue, domain:metadata, type Bug. A diagnostics endpoint whose failure mode is "0 problems" violates a contract this file states in its own prose (ADR-0110 D3: a miss and an outage are different facts); the acceptance set does not widen and the fix restores declared behaviour.

Dispatch guidance:

⛔ Not "delete the catch" — classify by error type through the existing isMissingTableError / rethrowUnlessMetadataStoreUnprovisioned discrimination, as the card says.
Between "propagate the 503" and "per-type degradation marker in the payload": propagating the 503 is the ruled default (ADR-0110 D3 restoration, no surface change). A new payload field is a public-surface addition — if the implementation concludes it is genuinely needed, stop and escalate rather than adding it under this card.
Run the written repro first and record prediction vs measurement.
The adjacent organizationId observation is explicitly not this card — leave it unmeasured here; it belongs to the tenant-scoping thread ([finding] The REST metadata write door stamps every audit row env-wide, so #8747's org scope cannot separate tenants on the REST-authored path #8805 family).

Size/model suggestion: M, opus.

Generated by Claude Code

Claim: PM loop
Session: session_01XeQRiAa7vYRVX5Fog7Zby8
Branch: claude/issue-8855-diagnostics-store-outage-503
Worktree: objectstack-issue-8855
Domain: domain:metadata
File surface: packages/metadata-protocol/src/protocol.ts — REGION getMetaDiagnostics ONLY (:4696 → :4819, on origin/main @ 8664a2c99), plus its pin files (stop on breach; explain in the report)
Container & model: M, mode:subagent, model: opus (triage's suggestion, adopted)
Serial constraints cleared: #8819 closed 12:53:19Z via merged PR #8870; two sibling protocol.ts cards dispatched concurrently in disjoint declared regions — #8868+#8833 at diffMetaItem (:15911), #8858 at publishPackageDrafts (:13570)

Implementing triage's ruling 5302846257 as written, ⛔ not re-adjudicated: classify by error type through the existing isMissingTableError / rethrowUnlessMetadataStoreUnprovisioned discrimination and propagate the 503. ⛔ Not "delete the catch". ⛔ A per-type degradation marker in the payload is a public-surface addition — if the implementation concludes it is genuinely needed, it stops and escalates rather than adding it here.

⚠️ Region boundary worth stating precisely. This card's region ends at :4819, the line where rethrowUnlessMetadataStoreUnprovisioned is declared. Calling that helper is in scope; modifying it is not — it is a shared producer with six other call sites (:5369, :5428, :5657, :5722, :6135, :10559, :11680), and a change there reaches two live sibling dispatches. If the fix seems to require editing the helper itself, ⛔ stop and report.

⛔ The adjacent organizationId observation on this card is explicitly out of scope (it belongs to the #8805 tenant-scoping thread) — leave it unmeasured, per triage.

Generated by Claude Code

{
  "issue": 8855,
  "status": "done",
  "branch": "claude/issue-8855-diagnostics-store-outage-503",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/8923",
  "premise_still_valid": true,
  "summary": "The card's premise is confirmed by measurement, not assumed. getMetaDiagnostics' per-type catch is narrowed (not deleted): a metadata-store 503 arriving from getMetaItems is rethrown UNCHANGED, every other failure still skips that one type. No payload field added, no surface change — the ruled default. The shared producer rethrowUnlessMetadataStoreUnprovisioned was NOT modified: the diff sits entirely inside getMetaDiagnostics (region :4696-:4819 respected; the helper's 7 other call sites and both live siblings are untouched). Notable design point: re-invoking that helper on what arrives here would be WRONG — isMissingTableError walks the cause chain, so it would re-wrap the already-shaped 503 in a second 503 and displace the driver error on `cause`. The envelope is propagated, not reclassified, and the pin asserts `cause` by identity to prove it. Two platform-written footers are on the PR body (mine plus a create-time appended duplicate) — platform behaviour, ground rule 2, not corrected.",
  "tests": "Union re-run AFTER the final commit, at HEAD b1de760d6 (post-merge; head.sha b1de760d6a9c68a3420331573bc1b758add31b68 confirmed on the PR).\n\nREPRO — prediction written down first, then measured; matched exactly. origin/main @ 8664a2c99, protocol-level:\n  [outage ECONNREFUSED]  RESOLVED total=0 scannedTypes=26 scannedItems=0 Object.keys(stats).length=0\n  [benign no such table] RESOLVED total=0 scannedTypes=26 scannedItems=0 Object.keys(stats).length=26, every stat {\"count\":0,\"locked\":0,\"packages\":[]}\nBoth harms confirmed: total:0 = '0 problems' during a total outage, and stats absent = the type vanishes from Studio tile counts. scannedTypes undecremented in both.\n\nABLATION — direction predicted BEFORE running (ordinary red, outage half ONLY): observed 3 failed / 3 passed, exactly as predicted. The 3 reds fail by RESOLVING with the defect payload verbatim: 'expected a rejection, but the sweep resolved with {\"entries\":[],\"total\":0,\"scannedTypes\":26,\"scannedItems\":0,\"stats\":{}}'. The 3 benign/selective/healthy controls stayed GREEN through the ablation — that separation is what shows the change is the outage split, not a blanket throw. Fix was committed BEFORE ablating; restored via `git checkout HEAD -- <path>`, porcelain clean after.\n\nDISCRIMINATING CONTROLS in the same pin file (6 cases): outage whole-corpus -> 503 + cause identity; outage single-type -> 503; outage vs first boot side by side; unprovisioned sys_metadata -> RESOLVES with every type present at count:0; a genuinely non-listable type -> RESOLVES, that one type skipped, the rest counted; healthy store -> real counts.\n\nSUITES: @objectstack/metadata-protocol 103 files / 1493 tests passed.\nCONSUMPTION RADIUS — DOWNSTREAM (prefix filter '...@objectstack/metadata-protocol' = consumers, 43 of them): ran the three that touch this surface — @objectstack/objectql 210 files / 3675 tests, @objectstack/rest 118 files / 1948 tests, @objectstack/client 23 files / 301 tests. All green. objectql carries diagnostics-clean-baseline.test.ts — the consumer named as having gone red on #8867.\n\nGATES: derived from actual changed paths via `node scripts/pm/dispatch-gates.mjs`, then ALL re-run at post-merge HEAD b1de760d6. Derived: check:changeset-gate-self-tests, check:cross-package-test-inputs, check:durability-log-level, check:filter-alias-parity, check:objectui-changeset, check-adr-0087-registration, check-changeset-no-major, check-empty-changeset. Convention-triggered (the script named these itself): check:query-options-erasure, check:type-check-coverage, check:type-check-debt --re-measure. ADDED BEYOND its list: check:nul-bytes, check:engine-double-contract, check:error-code-casing, check:adr-anchors. All PASS.\n\nTEST_DEBT measured DIRECTLY on a BUILT workspace (metadata-protocol is in the DEBT ledger with no typecheck script, so `pnpm typecheck` carries zero information about it): full workspace build (turbo, 70 tasks), then --re-measure re-ran tsc per ledger entry -> '33 ledger entries re-measured, 1926 raw tsc errors total, none above its recorded number'. metadata-protocol's recorded 63 unchanged, so the new test file contributes 0 errors. No ceiling raised. The lone -1 surplus is @objectstack/lint, pre-existing and untouched by this diff (gate calls it 'Not an error', tracked at #6376).",
  "open_questions": [],
  "out_of_scope_findings": [
    "filed as #8924: `scannedTypes` still overstates the swept set when a type is BENIGNLY skipped, and no payload field lets a caller detect partial coverage — the residue #8855's ruling deliberately parked (a degradation marker is a public-surface addition). Filed unassigned, `finding` + `domain:metadata`, NO pm:queue, so it is not lost when #8855 closes. Searched first; no existing card covers it."
  ]
}

Notes for the PM, outside the JSON:

Card body sanitizer check (asked for explicitly): the body is undamaged. It carries its own editorial note saying the first write lost two angle-bracket spans and was re-posted without them; the current body reads complete end-to-end through the closing editorial note, with no truncated prose. Nothing needed re-deriving.
The card's one UNVERIFIED claim, re-measured rather than trusted: the REST handler is at packages/rest/src/rest-server.ts:3478, not :4759 — refactor(rest): extract the ADR-0112 error/fault-classification prologue out of rest-server.ts #8887 moved it hours earlier. Substance holds: it returns the object verbatim via res.json(result) with no shaping. It also already wraps the call in try/catch → handleRouteError, which preserves status, so the 503 reaches the wire with no change in packages/rest. No widening into the domain:cli lane was needed.
organizationId left unmeasured, per the ruling — recorded in [finding] getMetaDiagnostics scannedTypes still overstates the swept set when a type is benignly skipped — no payload field lets a caller detect partial coverage #8924 only so nobody re-derives it as new.
Region discipline held. The diff touches getMetaDiagnostics only. No sibling region (diffMetaItem :15911, publishPackageDrafts :13570) was entered, and main merged cleanly with no conflict in protocol.ts.
No skip-changeset label: this PR ships a changeset (user-visible behaviour change), so the label would be wrong here.

Generated by Claude Code

fix(metadata-protocol): propagate the metadata-store 503 out of getMe…

fix(metadata-protocol): rethrow the request-classification 400 out of…

Navigation Menu

Repository navigation

[finding] getMetaDiagnostics swallows the 503 that #5532 raised — an unreadable metadata store is published as "0 problems", and the type vanishes from the Studio tile counts #8855

What it publishes

The swallow

Why the worst value is the benign one

`scannedTypes` is the field that would have caught it, and it is computed from the intent

What a fix must not do

Why this is its own card, not part of #6504

Reproduction — written, NOT run

Adjacent observation — UNVERIFIED, do not act on it from this card

Triage note (input, not a claim)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[finding] getMetaDiagnostics swallows the 503 that #5532 raised — an unreadable metadata store is published as "0 problems", and the type vanishes from the Studio tile counts #8855

Description

What it publishes

The swallow

Why the worst value is the benign one

scannedTypes is the field that would have caught it, and it is computed from the intent

What a fix must not do

Why this is its own card, not part of #6504

Reproduction — written, NOT run

Adjacent observation — UNVERIFIED, do not act on it from this card

Triage note (input, not a claim)

Activity

os-project-manager commented on Aug 15, 2026

hotlong commented on Aug 15, 2026

hotlong commented on Aug 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`scannedTypes` is the field that would have caught it, and it is computed from the intent