Skip to content

QA run · area:dashboards (9/9) · 2034975a · 2026-10-16 · 9 PASS / 0 PARTIAL / 0 FAIL / 0 BLOCKED / 0 NOT-RUN #23927

Closed

Description

@objectstack-fleet

Run record for selector area:dashboards, one slice of the #23878 wave (cloud runner for the #23878 runner session). Text only; screenshots were judgment aids and are not attached.

Environment fingerprint

key value
framework sha 2034975a47193f04db4791614f4ac9331ff211d3 (main, 2026-10-16 12:05 UTC)
.objectui-sha daeb77fc84dd824e74c95b87cdede85afdb248e7
console bundle objectui daeb77fc built against spec at a35f7a67^ (34b740f), the workaround for #23908 (pnpm objectui:build fails on main). Built in a separate worktree with scripts/build-console.sh, copied into a second worktree at 2034975a; pnpm check:console-sha there: "Console dist matches the objectui pin (objectui@daeb77fc84dd)". All browser verdicts below ran on that bundle.
boot OS_PORT=P pnpm exec objectstack dev --ui --seed-admin -p P -d file:DIR/data.db from the QA worktree's examples/app-showcase; API server :41871, browser server :41881, each with its own file DB
seed stock showcase seed + --seed-admin (admin@objectos.ai)
browser Playwright (global, /opt/node-tools) with executablePath: /opt/pw-browsers/chromium, viewport 1600x1100, one context per script, form sign-in
execution sequential, single runner, no subagents; one server at a time during vitest
clock runner UTC 2026-10-16 12:35–13:04; dashboard presets resolved last_90_days to 2026-07-18..2026-10-16
time ratio environment ≈ 18 min (install + build + console build), test ≈ 19 min, measured after the console bundle was ready. The API probes and vitest pins ran while the console was building, so they are not in either figure.

Scope

Selector area:dashboards → 9 runnable items. 0 blocked-hidden, 0 planned. Two ledger entries are retired and are not selected: dashboards.saved-report-ownership (rev 3) and dashboards.report-schedule-dispatch-delivery (rev 2), both retired by #20102.

item rev priority surface how judged
dashboards.strict-widget-rejects-stray-keys 2 P1 mixed API (12 variants + 4 chartConfig keys) + browser (Studio source editor, corrected widget)
dashboards.chart-first-paint 2 P2 browser browser + data API
dashboards.drill-through-range 2 P2 browser browser network capture + data API
dashboards.chart-type-matrix 4 P1 browser browser + data API + two vitest pins
dashboards.dataset-report-authoring 1 P1 mixed pin (dogfood) + API + browser
dashboards.empty-null-bucket-boundaries 1 P2 mixed two pins + browser + data API
dashboards.global-filters-rescope 4 P1 browser browser network capture + data API + Studio designer (D2)
dashboards.system-overview-live-counts 2 P2 mixed browser + data API
dashboards.cube-query 3 P2 api REST

Pin runs (rule 6)

All on the QA worktree at 2034975a, --maxWorkers=1, no server up:

  • packages/qa/dogfood: dashboard-designer-roundtrip.dogfood.test.ts, date-bucket-parity-conformance.test.ts, empty-group-bucket-parity.test.ts → Test Files 3 passed, Tests 15 passed.
  • examples/app-showcase: test/coverage.test.ts (incl. "covers every distinctly-renderable ChartType", "covers every report type"), test/dashboard-filter-vocabulary.test.ts → Test Files 2 passed, Tests 16 passed.
  • packages/spec: src/ui/dashboard-compareto.test.ts → Test Files 1 passed, Tests 17 passed.

Ground truth (data API, same boot)

POST /api/v1/data/showcase_task/query with group-bys on the stock seed:

  • status: 5 buckets × 2 tasks (backlog, todo, in_progress, in_review, done); estimate_hours by status 30 / 36 / 100 / 54 / 24, total 244.
  • priority: 4 buckets (high 3, low 1, medium 4, urgent 2).
  • created_at months: 2 (2026-09: 3, 2026-10: 7).
  • status × priority: 8 non-empty cells.
  • done:false → 8 tasks / 220 h; done:true → 2 tasks / 24 h.
  • showcase_project by account: 3 accounts; budget 800,000 / 200,000 / 90,000; spent total 616,000.

Per-clause verdicts

dashboards.strict-widget-rejects-stray-keys (rev 2) — PASS

# clause (short) verdict evidence
0 each legacy key rejected naming it pass PUT /api/v1/meta/dashboard/qa_stray_key_probe?mode=draft for each of the 11 LEGACY keys (object, categoryField, categoryGranularity, valueField, aggregate, aggregation, rowField, columnField, xAxisField, yAxisFields, measures) → 422 INVALID_METADATA, issue widgets.0 [unrecognized_keys]: "Unrecognized key(s) on this dashboard widget: `KEY`." with the literal key in each
1 error prescribes dataset + dimensions + values pass every legacy variant's issue text continues: "The pre-ADR-0021 inline analytics shape (…) was removed — bind a `dataset` and select `dimensions` + `values` by name. Renderer-only settings belong under `options`."
2 hallucinated key echoed, suggestion honest pass chartFlavour → 422, "Unrecognized key(s) on this dashboard widget: `chartFlavour`. Undeclared top-level keys were dropped silently before strict validation…". No did-you-mean appears, and no declared key is close to chartFlavour, so its absence is honest
3 wrong-layer chartConfig keys point at the right surface pass chartConfig.dataset → 422 "`dataset` is not part of the chart config — it is the dashboard widget's own key (ADR-0021), a sibling of `chartConfig`…"; chartConfig.drillDown → 422 "…it is a REACT-TIER prop, written beside `objectName`/`aggregate` on the ObjectChart component. On a DASHBOARD widget there is no per-widget drill configuration at all…"; aggregate and objectName → 422, each with its own guidance
4 rejected draft not persisted pass after each of the 16 rejected PUTs, GET /api/v1/meta/dashboard/qa_stray_key_probe → 404 RESOURCE_NOT_FOUND and no body carries the stray key. After the clean PUT (200) and publish (200), the GET body holds only id/type/title/dataset/dimensions/values. Browser door: in Setup → Metadata → Dashboard → qa_stray_key_probe → Edit → Source, adding "categoryField": "status" triggered the autosave PUT …?mode=draft → 422, the editor shows "This metadata does not match the spec — 1 validation error(s). Widgets → w1: Unrecognized key(s) on this dashboard widget: `categoryField`…", and the stored item still carries no categoryField
5 corrected widget renders pass the published qa_stray_key_probe (dataset showcase_task_metrics, dims [status], values [task_count]) draws 5 recharts bar rectangles with both axes on /_console/apps/com.example.showcase/dashboard/qa_stray_key_probe and in the designer canvas

dashboards.chart-first-paint (rev 2) — PASS

# clause verdict evidence
0 bars and axes on first paint, no resize pass cold load of showcase_chart_gallery and showcase_ops_dashboard at a fixed 1600x1100 viewport, screenshot after a 7 s settle, no resize: every chart widget shows its SVG marks and axes
1 bar count = distinct-status count pass Tasks by Status: 5 .recharts-bar-rectangle = 5 distinct statuses from the data API (both dashboards)
2 Delivery Operations KPI tiles show numbers pass Active Projects 2 (API: status=active → 2), At-Risk 1 (health=red → 1), Awaiting Review 2 (status=in_review → 2), Total Budget 1,090,000 (sum budget = 400k+90k+600k)
3 no swallowed query failure pass console capture for both loads: no error or pageerror entries. Every dataset query (18 on the gallery, 9 on ops) answered 200

dashboards.drill-through-range (rev 2) — PASS

# clause verdict evidence
0 date bucket drill scoped to that month pass click on the 2026-09 point of Chart Gallery line_created → GET /api/v1/data/showcase_task?…&filter=["and",["created_at",">=","2026-09-01T00:00:00.000Z"],["created_at","<","2026-10-01T00:00:00.000Z"]] (month bounds, not the year)
1 drilled count = bucket count pass status drill (Hours by Status → "In Progress") 2 rows = bucket 2; month drill 3 rows = 2026-09 bucket 3
2 matrix cell scoped by both dimensions pass Status × Priority cell In Progress × Urgent → filter=["and",["status","=","in_progress"],["priority","=","urgent"]], 1 row = API count for that cell (1)
3 strict subset pass 2 < 10, 3 < 10, 1 < 10 with ≥ 2 non-empty buckets in every aggregate
4 drawer with the bucket heading pass each drill opened the right-side drawer headed "In Progress" / "In Progress / Urgent" / "2026-09" with an "Open in list" link

dashboards.chart-type-matrix (rev 4) — PASS

Enum check (step 1): ChartTypeSchema has exactly the 20 listed members at 2034975a.

# clause verdict evidence
0 each of 20 variants renders, per variant pass per-widget screenshot of the 18 gallery cards plus 5 scratch tiles. bar, column, horizontal-bar: bar rects + axes; line: line curve + 2 x ticks; area: area path; combo: bars + line; pie and donut: sectors; funnel: trapezoids; scatter: symbols; radar: polygon; treemap: depth-1 rects; sankey: links + nodes; table and pivot: real HTML table elements; metric: 3 tiles. kpi, gauge, solid-gauge and bullet: scratch dashboard qa_single_value_gallery (PUT draft 200, publish 200)
1 mark counts = API buckets pass bar 5 = 5 statuses; column 4 = 4 priorities; horizontal-bar 5; pie 5 sectors; donut 4; funnel 5 trapezoids; radar 4 angle ticks = 4 priorities; line and area 2 x points = 2 month buckets
2 combo mixed marks, dual axes pass "Task Count vs Avg Progress": 2 bar rects (3, 7 = month counts) on the left axis 0–8, 1 line curve on the right axis 0–100 (≈82 → ≈32, consistent with the per-month avg progress from the seed), two .recharts-yAxis
3 treemap, sankey and scatter draw their own geometry pass treemap: 5 nested labelled rects; sankey: 6 nodes, 5 flow links; scatter: 6 points (= 6 distinct progress values)
4 single-value synonyms render the metric value pass kpi / gauge / solid-gauge / bullet tiles each read "10 Tasks", the same as the metric tile "10". The four look the same as each other and the metric tile, which the spec NOTE and SAME_AS_METRIC declare (value-only, no dial). Not scored against the "different family" negative
5 pivot is a true cross-tab pass "Tasks by Status × Priority": headers Status / Low / Medium / High / Urgent / Total, 5 status rows. Spot cells: Backlog×Low 1, Done×Medium 2, In Review×High 2, each equal to the API status×priority count
6 coverage ratchet pins the gallery pass coverage.test.ts "covers every distinctly-renderable ChartType" ✓
7 compareTo renders a comparison pass scratch qa_compareto_probe2 (dashboard dateRange created_at last_90_days; line on created_at + metric, both compareTo: { kind: 'previousPeriod' }): the wire selection carries compareTo and a timeDimension window 2026-07-18..2026-10-16, rows carry task_count__compare. The line draws a second dashed series "Tasks · vs previous period"; the metric reads "10 ↑100% vs previous period". Weak data: the previous window holds 0 tasks, so the comparison series is flat at 0. See fixture note F2
8 converged compareTo contract pinned at the parse pass dashboard-compareto.test.ts 17/17 ✓

dashboards.dataset-report-authoring (rev 1) — PASS

# clause verdict evidence
0 layout-less designer draft saves and publishes pass PUT /api/v1/meta/dashboard/qa_designer_probe?mode=draft (3 widgets, no layout) → 200; POST …/qa_designer_probe/publish → 200. Pin dashboard-designer-roundtrip.dogfood.test.ts 3/3 ✓
1 published form is ADR-0021 single form pass GET widgets: {id,type,title,dataset,values} and {…,dimensions} only; no object, categoryField, valueField or aggregate
2 layout-less widgets render pass qa_designer_probe draws all three: metric "10", bar 5 rects, donut 4 sectors (auto-flowed)
3 summary reconciles pass Hours by Status: Backlog 30.0, To Do 36.0, In Progress 100.0, In Review 54.0, Done 24.0, Total 244.0, each equal to the API sum by status
4 matrix reconciles pass showcase_status_priority_matrix (measure est_hours): Backlog×Low 12.0, To Do×Urgent 20.0, In Progress×Urgent 60.0, each equal to the API sum(estimate_hours) per cell
5 joined blocks partition pass Open Tasks block (runtimeFilter done:false) total 220.0 h; Completed Tasks block (done:true) shows Done 2 tasks. Partition from the API: done:false 8 tasks / 220 h + done:true 2 tasks / 24 h = 10 tasks / 244 h, the unfiltered totals. The two blocks carry different measures (hours vs count), so the clause's "sum of block totals" was reconciled per measure. See checklist note C2
6 embedded report chart plots the table's measure pass Hours by Status (Chart): 5 bar rects whose heights match the table 30/36/100/54/24 on a 0–100 axis. The wire shows two queryDataset calls (chart + table)
7 every ReportType per variant pass summary, matrix and joined rendered as reports above. tabular: /apps/com.example.showcase/showcase_task/view/tabular ("Task List") renders the Title/Project/Assignee/Status/Estimate grid of showcase_task (ADR-0021 Phase 2 ListView lens)
neg nested joined block rejected pass PUT /api/v1/meta/report/qa_joined_nested?mode=draft with blocks[0].type: "joined" → 422 blocks.0.type [invalid_value] 'expected one of "tabular"

dashboards.empty-null-bucket-boundaries (rev 1) — PASS

# clause verdict evidence
0 empty window gives a designed empty state pass Delivery Operations, custom window 2020-03-02..2020-03-08: KPI tiles 0 / 0 / 0 / 0; every chart and table reads "No data yet — This widget loaded successfully and its query returned no records yet. Source: …". No stale values and no console errors
1 emptiness is server truth pass all 9 re-issued dataset queries carry created_at {$gte:"2020-03-02",$lte:"2020-03-08"}; responses are rows:[] or count 0
2 KPI tiles 0 for the empty window pass tile values 0 match the queries' project_count:0, task_count:0 and budget_sum:0
3 date-bucket parity pass date-bucket-parity-conformance.test.ts ✓ (driver-sql + driver-sqlite-wasm, plus the negative detectors)
4 NULL bucket label parity pass empty-group-bucket-parity.test.ts ✓. Live check: showcase_account group-by signed_on via /data → signed_on: null (n=3); the same group via /analytics/dataset/query → signed_on: null (3). One shape on both doors
5 single datapoint renders, annotated pass (single-datapoint) task_status = To Do: Task Throughput draws a lone marker at 2026-10 = 2 (both To Do tasks were created in October). This is single-datapoint evidence. Unfiltered, line_created draws 2 points = 2 seeded months, so there is no collapse to one bucket

dashboards.global-filters-rescope (rev 4) — PASS

# clause verdict evidence
0 region re-queries map per widget field pass Revenue Pulse, Region → EMEA: invoice queries carry {"region":"emea"}, account queries (kpi_new_accounts, col_accounts_by_month, bar_accounts_by_industry) carry {"sales_region":"emea"}, each ANDed with its date bound
1 dateRange maps per widget date field pass custom 2026-08-01..2026-09-30: invoice widgets issued_on {$gte:"2026-08-01",$lte:"2026-09-30"}, account widgets signed_on {…}
2 opted-out KPI does not re-query or change pass "Accounts (all time)" (filterBindings false) issued its unfiltered query only on load; no query for it after the region change, the custom window or Reset; its value is 14 throughout
3 displayed aggregate = server truth pass under EMEA + last 90 days: Invoices tile 1 = API count (region emea, issued_on window) 1; Accounts Signed 1 = API count (sales_region emea, signed_on window) 1. Under 2026-08-01..09-30: Invoices 4 = API 4; Accounts Signed 2 = API 2
4 date-scoped account charts exclude NULL signed_on pass 3 accounts have NULL signed_on (API). Date-scoped account count under the custom window is 2 = API count with the same signed_on bounds, and the NULL rows are not in it (all-time tile 14)
5 Delivery Ops intersection + opted-out project tiles pass Task Status = In Review: kpi_awaiting_review query $and:[{status:in_review},{created_at…},{status:in_review}] → 2 (API in_review = 2). Task Status = To Do → 0 (in_review ∧ todo is empty). Active Projects 2, At-Risk 1, Total Budget 1,090,000 unchanged, and no re-query for them. Pin dashboard-filter-vocabulary.test.ts ✓
6 D2 Studio inspector authors filterBindings pass scratch package com.qa.dashprobe (POST /api/v1/packages 201) holding qa_filters_probe. In Setup → Metadata → Dashboard → qa_filters_probe → Edit, inspector "Dashboard Filter Bindings": unticked Apply for Region on acc_count, re-targeted Region to "Lifecycle" (status) on acc_by_industry, Save, Publish. Wire: PUT …?mode=draft 200 with "filterBindings":{"region":false} / {"region":"status"}, then POST …/publish 200. GET after publish carries both maps. On reload with Region = EMEA: acc_by_industry re-queries with runtimeFilter {"status":"emea"} (rows [], designed empty state) and acc_count issues no query (stays 14)
7 D1 Reset is dirty-gated pass pristine bar: 0 Reset buttons and 0 .lucide-rotate-ccw. After Region → EMEA: 1 of each. Reset → bar back to "Last 90 days · Region: All", Reset gone, 9 re-queries with the default window
8 D3 {value,label} options + server group-by pass Revenue Pulse static options All/AMER/EMEA/APAC; the trigger shows "EMEA" and the committed value is emea (wire). Scratch optionsFrom {object: showcase_account, valueField/labelField: sales_region}: the option list comes from POST /api/v1/analytics/dataset/query with an inline dashboard_filter_options dataset, GROUP BY sales_region, limit 1000 → AMER 9 / APAC 1 / EMEA 4, raw values amer/apac/emea in drillRawRows. Choosing EMEA commits {"sales_region":"emea"} → 4 accounts

dashboards.system-overview-live-counts (rev 2) — PASS

# clause verdict evidence
0 Total Users = COUNT(sys_user) pass tile 3; /data/sys_user count 3; wire sys_user_metrics → 3
1 Active Sessions = live-session count pass (weak) tile 34; the wire query carries {revoked_at:null, expires_at:{$gt:"2026-10-16T13:01:06Z"}} → 34; the direct /data/sys_session count with the same predicate → 34. The unfiltered count is also 34: no revoked or expired session existed in this boot, so the predicate could not be shown to discriminate (fixture note F3)
2 widget query on the wire matches pass sys_user_metrics → 3 and sys_session_metrics → 34, equal to the tile and the direct aggregate
3 single-environment widgets hide pass GET /api/v1/meta/dashboard/system_overview serves 8 widgets: widget_organizations is absent (server strip), widget_packages_installed is present (requiresObject: sys_package_installation). The board renders 7 widget cards, neither gated widget takes a slot, and there is no blank tile
4 no blank widget pass 4 metric tiles numeric (3 / 34 / 34 / 2); Audit Events by Action pie 5 sectors; Events by User bar 2 rects ("(None)" 89, Dev Admin 68); Event Volume table 5 rows + Total 157
5 distribution = distinct actions pass pie 5 segments and table 5 rows = 5 distinct sys_audit_log.action values from the direct group-by (config_change, create, delete, login, update)

dashboards.cube-query (rev 3) — PASS

# clause verdict evidence
0 meta discovers the cube pass GET /api/v1/analytics/meta → 200; ?cube=showcase_delivery → data[0] with measures showcase_delivery.count (count), .total_estimate_hours (sum), .avg_estimate_hours (avg) and dimensions .status, .priority, .due_date (time), .assignee. The wire is a bare array under data, not cubes[] (checklist note C1)
1 count by status reconciles pass cube rows backlog 2 / done 2 / in_progress 2 / in_review 2 / todo 2 = /data/showcase_task/query group-by status count
2 total_estimate_hours reconciles pass cube 30 / 24 / 100 / 54 / 36 = /data sum(estimate_hours) by status. avg variant: 15 / 12 / 50 / 27 / 18 = /data avg
3 rides the caller's read scope pass (structural) admin cube aggregate = admin /data aggregate. Mechanism: packages/runtime/src/domains/analytics.ts forwards the request ExecutionContext, and service-analytics composes a per-object read scope (read-scope-sql.ts). showcase_task is public_read_write, so no cross-persona discrimination is possible on this fixture (cross-persona proof lives in search / access-security, per the clause)
4 degradation pass (served-not-absent) slot is filled on stock showcase (meta 200 above). The absent-slot 404 arm was not exercised; per the clause, this is recorded as served-not-absent
5 malformed query rejected at entry pass {cube, filters:{status:"done"}} → 400 VALIDATION_FAILED "`filters` is not an AnalyticsQuery field — use `where`…". {cube, query:{…}} → 400 "`query` was removed from AnalyticsQueryRequest in @objectstack/spec 17.0.0 … Move the query.* fields to the body top level…"

Reproduction rules

No fail verdicts this run, so there are no reproduction rules to give.

Derived item verdicts

item verdict
dashboards.strict-widget-rejects-stray-keys PASS
dashboards.chart-first-paint PASS
dashboards.drill-through-range PASS
dashboards.chart-type-matrix PASS
dashboards.dataset-report-authoring PASS
dashboards.empty-null-bucket-boundaries PASS
dashboards.global-filters-rescope PASS
dashboards.system-overview-live-counts PASS
dashboards.cube-query PASS

Fixture gaps and checklist notes (closed out on #23878, not extracted)

  • F1 — chart-type-matrix fixture requires[2] (qa_compareto_probe) omits that the dashboard needs a dateRange. As written (time dimension + compareTo, no dateRange), the widget renders the named error "Dataset query failed: 400 … compareTo needs a dated window to shift, but this selection declares no timeDimension with a dateRange". That error is loud and correct, not a product fail. Adding dateRange: {field: 'created_at', defaultRange: 'last_90_days'} makes it render.
  • F2 — the compareTo probe on the stock seed compares against an empty previous window (all tasks are created within the last ~45 days), so the comparison series is flat at 0. A seed with tasks older than 90 days would make the clause discriminate.
  • F3 — system-overview-live-counts clause 1 cannot discriminate on a fresh boot: no revoked or expired sys_session rows exist, so the filtered and unfiltered counts are equal. The item needs a step that revokes or expires one session first.
  • C1 — cube-query clause 0's verify names "the meta response's cubes[] entry". The wire contract is a bare array under data (packages/spec/src/api/analytics.zod.ts says explicitly that there is no cubes wrapper).
  • C2 — dataset-report-authoring clause 5 asks for "open_block totals + done_block totals == unfiltered totals", but the two blocks carry different measures (est_hours vs task_count). The partition can only be reconciled per measure through the API.

Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Close-out for this record (checklist-test §5).

    Extracted: #23931 · dashboards.dataset-report-authoring · matrix is a true pivot whose cells reconcile

    No fail verdict this run, so no disposition is owed. #23931 is an incidental user-visible defect seen while judging that clause, which passed on its cell values. The fixture gaps (F1–F3) and checklist notes (C1, C2) in the body close out on the wave anchor #23878.


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Closing at 2026-10-16T14:21Z: this record's §5 close-out is already posted — comment 6216188189 (Extracted: #23931 · dashboards.dataset-report-authoring), with 9 pass · 0 partial · 0 fail of 9. Its fixture gaps F1–F3 and checklist-accuracy findings C1–C2 close out on the wave anchor #23878; the zero-baseline delta it handed over is objectstack-ai/objectui#12677.

    Closed by the domain:devx seat 1 PM (seat post #6023) · os-sales · session_014eFvbvGK4FjXvscggDNTD4, as part of the paced close-out of the #23878 wave.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions