Skip to content

v17 verification tracker — browser-verify every user-facing change in the 17.0.0 train #3909

Closed

Description

@os-zhuang

Tracking issue for end-to-end browser verification of the v17 (17.0.0-rc) train, per the release plan. A dedicated agent will work through this checklist against a running app (PORT=3000 pnpm dev showcase + :5180 objectui dev server where UI-side, or the dogfood-verification flow), acting as a real admin/user. Check items off as they are verified; file a separate issue for every failure found and link it here.

Defect rollups: #4482 (rc.0/rc.1 — 17 defects, all closed) · #5145 (rc.2/rc.3 — 3 defects, 2 open). Individual issues are also linked from the item they block, below.

Sources of truth: the curated release page content/docs/releases/v17.mdx, the 300+ pending changesets in .changeset/, and the objectui window cf2d56e32a11 → 4a4829d0ef39 (incl. the backfilled range 2cb8d78e24ad...c6cfdf1288b6, see PR #3908). The docs-sweep log lives at docs/v17-docs-sweep.md.

A. Breaking behavior — verify the new posture actually enforces

B. Removed surfaces — verify they are actually gone (and loud)

C. New capabilities — verify they work end-to-end

D. Docs verification (site)

E. Known out-of-scope items already logged

  • Spec comment drift: core-services.zod.ts 'ai' member comment still says "NLQ, Chat, Suggest, Insights" (comment-only, spec-touching PR).
  • content/docs/releases/implementation-status.mdx predates the v17 window; needs its own regeneration pass.
  • RC exit: after changeset pre exit, re-check the v17 page's version wording and this checklist's remaining items.

F. 17.0.0-rc.1 supplement

Sections A–E were written against the rc.0 cut (watermark a641d10, just after the rc.0 version commit fc156fa4a). This section adds what shipped in 17.0.0-rc.1 (version commit 2bafe62e8, 2026-07-31): the window a641d10 → 2bafe62e8, ~340 non-release commits, objectui pin advanced 4a4829d0ef39 → e651c936870e → a136322f8723 → bebaebd39ace → 7d9734d5e321. (Commits after 2bafe62e8 — #4421/#4423/#4424/#4425/#4427/#4429 — are post-rc.1 and belong to the next rc's checklist.) Same rules: check off what passes, file a separate issue per failure and link it here.

F1. Actions — identity, dispatch, and failure semantics (ADR-0110 / ADR-0112)

F2. Sharing & security hardening (ADR-0111)

F3. Query/write strictness — silent drops became refusals

F4. Temporal correctness (ADR-0053 D-A3/D-D)

F5. Analytics honesty

F6. New capabilities since the rc.0 cut

F7. Boot / CLI / authoring strictness

F8. Docs (run 2+)

  • Re-run the docs sweep per docs/v17-docs-sweep.md from watermark a641d10 → 2bafe62e8; append the run entry.
  • At changeset pre exit: re-check the v17 release page wording and close out sections A–F.

G. 17.0.0-rc.2 / rc.3 supplement

Sections A–E cover rc.0, F covers rc.1. This section covers the window 2bafe62e8 (rc.1) → a1a855a28 — 277 non-release commits, 283 new changesets, objectui pin 7d9734d5e321 → f5bc4c78be76. Two release cuts fall inside it: rc.2 (89d2a4eb3) and rc.3 (cut at 2e284b254, published as the npm rc dist-tag; main has since moved 56 commits past that point, and main's package.json still reads 17.0.0-rc.2 because the version commit was not merged back).

All 17 defects filed from the A–F verification are closed; G1 re-verifies them rather than trusting the close.

Three new ADRs land in this window: ADR-0118 (non-user actor contract), ADR-0119 (reachable transactions + honest atomic batch), ADR-0121 (declarative-endpoint routing, rules only — implementation tracked separately).

G1. Regression — the A–F defects stay fixed

G2. ADR-0118 — the non-user actor contract

G3. ADR-0119 — atomic means atomic, or it refuses

G4. Hook / validation evaluation — the #4649 family

G5. Metadata plane honesty

G6. Seeds / boot / driver

G7. Authorization surfaces

G8. The #4001 / ADR-0049 close-out wave

This window is dominated by spec surface convergence — dual-source name collapses and inert-key retirements. Most are compile-time; verify the ones with a runtime face:

G9. Client / React

G10. Docs & close-out

Activity

  1. self-assigned this
    on Jul 30, 2026
  2. self-assigned this
    on Aug 1, 2026
  3. baozhoutao commented on Aug 1, 2026

    @baozhoutao
    Contributor

    rc.1 verification run 1 — section F

    Environment: framework main @ 1ee48bc60 (post-rc.1), showcase example under os serve --dev --ui on an isolated SQLite datastore, @objectstack/console@17.0.0-rc.1 (the published vendored SPA, pin 7d9734d5e321) at /_console/, Node 22.22. Driven as admin@objectos.ai plus a second signed-up member for every two-principal check.

    19 items ticked. Highlights of what actually held up under a live request:

    • Actions (F1). An undeclared action name is refused with the prescriptive ADR-0110 404; script runs, flow dispatches (returns status:"paused" with the screen payload), and a url/modal action reaching the server is refused with a message naming its client-side nature. Param enforcement is strict by default and precise: missing required → 400 naming the param, unknown key → "Unknown action param … not declared on this action", bad option → the allowed list. Field validation localizes properly (Accept-Language: zh-CN → 优先级必须是以下值之一) and carries per-field fields[] with labels.
    • Sharing (F2). The ADR-0111 verb boundary is real: an edit-level share lets the recipient read and PATCH, and refuses DELETE (403) and share-management ("requires record ownership or Modify All Data (ADR-0111 D1)"). A criteria-less rule is refused with the ADR-0049 message. A non-participant cannot read an approval request (404). Anonymous data access is denied on both read and write, while audience:'public' book metadata is served without a session.
    • Query strictness (F3). Every silent-drop this train closed refuses correctly, each with an actionable message: unknown query param, sort/select/expand/dotted-path sort naming a missing field, non-AST $filter. Implicit field filters AND-merge with an explicit filter (verified by counts: 2 ∧ 1 → 1). deleteMany is ids-only (a where body is rejected), the batch cap fires at 200, and sorted and unsorted paging are exact partitions (page union == full set, no dupes/gaps).
    • Temporal (F4). +08:00 input stores as UTC …T10:30:00.000Z, time stores wall-clock 09:15:30, and a bare-day upper bound ($lte: '2026-08-15') covers the whole day on both date and datetime.
    • Analytics (F5). /analytics validates at the entry (unknown body key rejected by name), a sum measure emits SUM(total) not COUNT(*), $or/$not reach the SQL, and a monthly timeDimensions bucket projects into rows in date order.
    • New capabilities (F6). The multi-value lookup seed resolves (f_lookups: ['Northwind','Contoso'] → a two-id array). required enforces on write with per-field errors, and the Console form shows the same three fields as required. The approval chain worked end to end in the browser and over REST: submit → request created → structured reassign to a second user → that user approves → status: approved, and sys_approval_action attributes each decision to the acting user (submit/reassign → admin, approve → member). The screen flow ran from the list toolbar, collected input, and the write landed (assignee = the value typed in the dialog).
    • Boot/CLI (F7). os serve boots a stack with no compiled artifact; the undeclared-stack-key lint reports each key with a remedy (including the storage → OS_STORAGE_* guidance); rc.1 tarballs carry CHANGELOG.md and a files whitelist (@objectstack/client ships 10 files).

    Failures filed (7)

    # Area What
    #4431 actions a sandbox capability denial answers 400 as a deliberate rejection — the crash contract (#3951) says 500 — and leaks the SandboxError: debug prefix
    #4432 metadata /meta/<type> overlay writes don't normalize the type segment: plural vs singular land in different namespaces, one overlay row shadows the whole code-authored listing, and the row is undeletable
    #4433 sharing deactivating a rule never withdraws its materialized grants — not on record touch, not after a full restart
    #4434 sharing DELETE /sharing/rules/:idOrName 500s for both address forms — no sharing rule can be deleted over REST (compounds #4433: an over-granting rule is unrecoverable)
    #4435 data PATCH/DELETE of a nonexistent record answer 200 success (record: null / success: true) where GET correctly 404s; deleteMany reports every typo'd id as deleted
    #4436 data the unsupported-filter-operator refusal ships without error.code and leaks the [sql-driver] prefix (ADR-0112)
    #4437 analytics a measure naming a missing field 500s with SQLITE_ERROR instead of a 400 naming the field

    Notes for the next run

    • Two toast strings in the Console are still English on a zh-CN session ("This flow runs on a single record — select a row first.", "Done") — worth a pass against the objectui i18n packs before RC exit rather than filing per-string.
    • The showcase's own dev script resolves a stale global @objectstack/cli@9.0.0 from the user PATH instead of the workspace CLI, which fails the boot with a misleading "plugin-approvals … not installed". Not a platform defect; worth a line in the contributor docs since it costs a debugging cycle.
    • Not yet covered in this run and still open in section F: the file/attachment axis (F6 #4208/#4285 gate), flow run summaries (selected/acted/skipped), os migrate value-shapes, and the F8 docs sweep.
  4. baozhoutao commented on Aug 1, 2026

    @baozhoutao
    Contributor

    rc.0 verification run 2 — sections A / B / C

    Run 1 covered only the F supplement. This run works the rc.0 checklist (A–D) on the same environment (framework main @ 1ee48bc60, showcase under os serve --dev --ui, published @objectstack/console@17.0.0-rc.1, isolated SQLite, admin + a second signed-up member). 7 more items ticked — 26 of 59 total. Three new defects filed.

    A. Breaking behavior

    B. Removed surfaces

    • SDK removals verified by instantiating the client, not by grep (grep is misleading here — nlq/suggest/insights still appear in comments documenting their removal). On a live ObjectStackClient: permissions, realtime, workflow, views, projects.listTemplates are all undefined; client.ai exposes only chat, chatStream, complete, models, conversations, agents, pendingActions; client.notifications exposes only list, markRead, markAllRead (no device/preference delegates).
    • Marketplace move (POST /api/v1/packages is a publish-vs-install shape collision — REST registers first and shadows the dispatcher (#3587 finding) #3610). POST /packages is the install surface again ({"code":"VALIDATION_ERROR","message":"Package id is required"} — reachable, no collision).
    • /graphql → 404 ✅, but discovery still advertises the slot: services.graphql = {"enabled":false,"status":"unavailable","message":"No implementation ships for the 'graphql' slot — register a service under it to enable"}. That message invites someone to fill a slot for a surface v17 deliberately removed, and it contradicts the docs sweep's own run-1 fix (services-checklist was reduced to 16 services with every graphql row deleted). Left unticked pending a call on whether the slot should be dropped from CORE_SERVICE_PROVIDERS or the message reworded.
    • Not verifiable here: POST /packages/publish answers 405 (Allowed: DELETE, GET, HEAD, PATCH) — the route-manager's /packages/:id matches with id="publish" and shadows it. The ledger declares the publish route as source: 'direct-mount', disposition: 'server-only', so it is presumably registered only where the marketplace publish plugin is loaded. Worth confirming on a deployment that has it — the shadowing means route-registration ORDER decides the winner.

    New defects (3)

    # Area What
    #4441 data / RBAC a lookup accepts an id that does not exist in the referenced object — including sys_position_permission_set.permission_set_id, so a grant row can point at nothing while the audience-anchor gate has to resolve that set to judge it. Found by accident: a shell variable expanded to FAIL and the platform stored it against the everyone position.
    #4447 data created_at is client-writable on an ordinary PATCH and persists (1999-01-01 written and read back). updated_at and created_by are correctly protected — this is one field of the audit trio. Its metadata says readonly: false.
    — — (the two #3909 F-section notes from run 1 stand)

    While chasing #4447 I could not find any input on this build that produces a non-empty droppedFields: computed fields (f_formula, f_summary) are rejected 400 INVALID_FIELD, unknown fields 400, and system fields are simply written. The #3794 contract may only have live coverage on the FLS path — worth a targeted check before RC exit rather than assuming the C-section item passes.

    Remaining

    33 unchecked. The bulk needs setup this run did not build: a runAs:'user' record-change flow fired by a system write (#3760), a per-tenant unique specimen (#3696), multi-org / membership-grade (ADR-0108, ADR-0105), better-auth 1.7 account migration, datasource fail-fast (#3741), the files-as-records axis (ADR-0104), historical import, and the Console-side items (objectui#28xx). Section D (docs) is untouched.

  5. baozhoutao commented on Aug 1, 2026

    @baozhoutao
    Contributor

    rc.0 + rc.1 verification run 3 — the remaining sections

    Continuing from runs 1–2. Same environment; this run built a dedicated probe stack (examples/rc1-probe, removed afterwards) for the items the showcase has no specimen for, plus a synthetic pre-v17 datastore for the migration items.

    40 of 59 ticked (was 26). Five new defects filed, one of them the most serious found across all three runs.

    🔴 #4467 — /analytics/query ignores record-level scoping

    A plain member with shares on 2 of 5 private notes:

    GET  /data/showcase_private_note              → total 2      ✅
    POST /analytics/query {cube, measures:[count]} → count 5      ❌
    POST /analytics/query {…, dimensions:["title"]} →
       returns the TITLES of all 5, including the 3 they cannot read
    

    Any authenticated caller who can reach /analytics can enumerate field values of every row of any cube-exposed object, past OWD, sharing rules and RLS. The other analytics items on the checklist (#4019 capability mount, #4010 entry validation, #4184 declared aggregates, #3650 dateRange, #3921 time ordering) all hold — the scoping layer is the piece that is absent; the returned data.sql carries only the caller's own predicates and no owner/share filter.

    Method warning for whoever re-tests: a viewAllRecords set left bound from an earlier test makes both paths agree at 5/5 and look correct. Confirm the data path returns fewer rows than admin before reading anything into the analytics numbers — I got a false negative from exactly this.

    Other new defects

    # What
    #4462 an object mapped to an unreachable datasource silently reads/writes the DEFAULT store — boot succeeds, broken never appears in the log, /ready 200, listing says status:"unvalidated". Against all three claims of the A5 item.
    #4466 an admin override of a properly-staffed approver slate is recorded as an ordinary approval — sys_approval_action has no override column at all, so bypassing designated approvers is byte-identical to being one.
    #4455 os migrate value-shapes misses a lookup holding an embedded record object — the exact case its own docstring names — because ReferenceIdValueSchema is z.string().min(1). The gate then closes on evidence it never gathered.
    #4441 a lookup accepts an id that exists nowhere, including the RBAC permission-set link tables.

    Verified this run

    Two checklist-wording corrections

    1. Seeds. The item promises Seeds: X inserted · Y updated · Z skipped. That shape does not exist — printSeedSummary deliberately collapses the three into one N rows / N ok / M errors count and uses · to separate sources. The code's contract is coherent; the checklist text is stale.
    2. os migrate meta. "rewrites all three" reads as an in-place codemod. It rewrites the stack snapshot (--out), not your TypeScript sources.

    Still open (19)

    Mostly Console-side objectui items needing UI-level assertions (API console AI family, dropped-field toasts, quorum chips, historical-import wizard), better-auth 1.7 account migration (needs pre-existing social/SSO rows), the four items blocked on the defects above, and the two close-out items that can only run at changeset pre exit.

    Also unresolved from run 2: I still cannot produce a non-empty droppedFields on any input — computed fields 400, unknown fields 400, system fields are written. #3794's contract may only have live coverage on the FLS path.

  6. baozhoutao commented on Aug 1, 2026

    @baozhoutao
    Contributor

    Final run — the checklist is worked through

    48 of 59 verified. The remaining 11 are now annotated inline: 10 are blocked on defects filed from this verification (they cannot be ticked until a fix lands), and 1 (changeset pre exit close-out) can only run after RC exit. Nothing is left untested for lack of effort.

    Four more defects this round, on top of the nine already filed.

    🔴 #4477 — screen-flow resume performs no server-side validation

    A two-field screen node (kind required; escalation_reason required + visibleWhen: "kind == 'escalate'"):

    resumed with expected actual
    {"kind":"normal"} — conditional field hidden accept 200 ✅
    {"kind":"escalate"} — field VISIBLE, required, missing reject 200 ❌
    {} — kind itself required, missing reject 200 ❌
    {"kind":"normal","totally_bogus":"x"} — undeclared key reject 200 ❌

    The render half is fine — the screen payload carries visibleWhen intact. But "honored in validation" passes only vacuously: nothing validates, so the conditional case cannot be got wrong. A client that posts straight to resume bypasses every required the author declared. This is the one gap in an otherwise consistent row — action params (#4213), record writes (ADR-0113), and approval decisionOutputs (#3447) all enforce their declared contracts, verified working.

    #4475 — every Setup System Overview tile reads 0

    Users 0 (actually 4), Sessions 0 (actually 16), all event counts 0. The widget's own response gives the cause:

    {"rows":[{"user_count":0}],
     "sql":"SELECT COUNT(*) AS \"user_count\" FROM \"sys_user\" WHERE created_at = $1"}

    An "all time" range is lowered to WHERE created_at = $1 — an equality test on a timestamp. The cube path lowers ranges correctly (>= $1 AND < $2), so this is specific to the dataset path. First screen an operator sees, every number wrong, no error surfaced.

    #4476 — docs date the #4286 removals to spec 18; they ship in 17

    Nine passages across three hand-written pages say query.cursor / distinct / joins / windowFunctions were "removed in @objectstack/spec 18". PROTOCOL_VERSION = '17.0.0', the package is 17.0.0-rc.1, and os migrate meta --from 16 labels all four [protocol 17]. A reader on 16 is told the break is a major away and plans accordingly. Found by the sweep run the tracker asks for — the sweep's fingerprint list should gain a version-number check, not just surface names.

    #4478 — quorum progress and per-group chips do not render

    The server publishes behavior:"quorum", minApprovals:2, per-approver group labels, and even a pre-rendered __nodeLabel: "Committee Sign-off (2 of 3)". The Console renders none of it: scraping the approval panel gives {lockBadge: true, approveBtn: true, quorumHint: false, groupChips: false}. An approver on a quorum node cannot see how many more approvals are needed. The lockRecord half of the same line does render (审批中已锁定 badge), so the panel is otherwise correct — this is specifically the progress surface. Fix belongs in objectui #2811; no framework change needed, the data is already on the wire.

    Verified this round

    One open judgement call for the maintainers

    /graphql correctly 404s, but discovery still reports a graphql service slot: {"enabled":false,"status":"unavailable","message":"No implementation ships for the 'graphql' slot — register a service under it to enable"}. That message invites someone to fill a slot for a surface v17 deliberately removed, and it contradicts the docs sweep's own run-1 fix (services-checklist reduced to 16 services with every graphql row deleted). Left unticked: whether to drop the slot from CORE_SERVICE_PROVIDERS or reword the remedy is a design call, not a test result.

    All 13 defects from this verification

    Security / correctness: #4467 (analytics ignores record scoping — reads values a caller cannot read), #4433 + #4434 (a sharing rule's grants cannot be withdrawn and the rule cannot be deleted), #4447 (created_at forgeable), #4441 (dangling lookup references incl. RBAC link tables), #4462 (unreachable datasource silently uses the default store), #4477 (screen resume unvalidated), #4466 (admin override unrecorded).

    Contract / envelope: #4431 (capability denial 400 vs the 500 the crash contract promises), #4435 (missing-record writes report success), #4436 (filter-operator refusal has no error.code), #4437 (analytics measure 500s with SQLITE_ERROR), #4432 (meta overlay type-segment phantom rows), #4455 (value-shapes gate misses its own headline case).

    UI / docs: #4475 (Setup overview all zeros), #4478 (quorum chips), #4476 (docs version drift).

  7. baozhoutao commented on Aug 1, 2026

    @baozhoutao
    Contributor

    All defects from this verification are now rolled up in #4482 — 17 issues grouped as security/data-integrity (8), contract/envelope (6), UI/docs (3), with a suggested RC-exit triage and the two non-defect judgement calls (/graphql discovery slot; two stale checklist claims).

    Note a correction to my run-4 comment above: it said "13 defects". The grouped list was right, the total was a miscount — it is 17.

  8. added
    trackingGenerated anchor/tracker issue — not dispatchable; exempt from the half-state sweep's findings
    domain:cliRetired 2026-10-22, alias of domain:server (cli surface); in-flight cards keep it until they land
    target:v17Release blocker for v17 — on the board until fixed, dropped as no longer valid, or accepted for GA
    on Aug 3, 2026
  9. baozhoutao commented on Aug 4, 2026

    @baozhoutao
    Contributor

    rc.3 verification — G1 regression + G2–G10 first pass

    Environment: main @ a1a855a28 (the rc.3 train tip — note rc.3's version commit c6a52d3a4 was tagged and published to the npm rc dist-tag but not merged back, so main's package.json still reads 17.0.0-rc.2 while main sits 56 commits past the rc.3 cut). Real showcase boot, two live principals, purpose-built probe stacks for the rest.

    73 of 113 ticked. Two new defects.

    G1 — 16 of 17 fixes verified landed, 1 residual

    I re-ran every original repro rather than trusting the close. The fixes are real and several are better than what I asked for:

    Residual → #5079. #4432's namespace normalisation, shadowing and immediate dispatch are all fixed, but after DELETE the overlay stays in the /meta listing while dispatch correctly 404s. The delete is real (reset:true, no sys_metadata row); only the enumeration view is stale — the mirror of the seam #4521 fixed on the write side.

    G2–G10 — what passed

    New defect → #5086

    allowRuntimeCreate: false is not enforced. #4509 set the flag on job and its changeset says "no 'create job' in Studio or via PUT /meta". It still creates one:

    PUT /api/v1/meta/job/rc3_runtime_job {"schedule":{"type":"cron","expression":"0 0 * * *"},"handler":"nope"}
    → 200 "Saved customization overlay (env-wide) — type=job"     … and it lists, with handler "nope"
    

    agent, the only other flagged type, behaves the same. supportsOverlay: false looks equally unenforced — the success message calls the row a "customization overlay" for a type declared not to support them.

    This is the ADR-0049 failure mode one level up: the enforcement flag itself is the silently-inert declaration. It is easy to miss because a minimal payload 422s on schema validation first — only a payload that passes the schema reveals that the gate never runs. Studio presumably reads the flag to hide "create", so the UI honours a rule the API does not.

    Remaining 40

    Mostly G5 (metadata-plane honesty: publishDraft drain, DDL loudness, history/event_seq invention), G9 (client-react hook loops — needs a React harness), the G8 spec-rename family (compile-time, best covered by a typecheck of a consumer rather than a runtime probe), and the close-out item that can only run at changeset pre exit.

  10. baozhoutao commented on Aug 4, 2026

    @baozhoutao
    Contributor

    rc.3 verification — complete

    108 of 113 verified. The 5 open items are: 2 blocked on defects filed from this pass (#5086, #5088), 2 close-out items that can only run after changeset pre exit, and the #4432 residual tracked in #5079.

    Three new defects this round, on top of the seventeen from the A–F passes (all of which are now closed and re-verified as genuinely fixed, not merely closed).

    New defects

    # What
    #5086 allowRuntimeCreate: false is not enforced. PUT /meta creates job and agent items the registry declares code-only. supportsOverlay: false looks equally unenforced — the success message calls the row a "customization overlay" for a type declared not to support them. This is the ADR-0049 failure mode one level up: the enforcement flag itself is the silently-inert declaration.
    #5088 updateMany runs hooks for a nonexistent id. With no stored record to merge, the hook condition evaluates against a payload-only record, #4775's abort fires, and the row fails INTERNAL_ERROR accusing a correct hook of naming an undeclared field — where single-record PATCH correctly answers RECORD_NOT_FOUND. Under atomic one stale id poisons the whole batch via NOT_ATTEMPTED.
    #5079 #4432 residual — a deleted overlay stays in the /meta listing while dispatch correctly 404s.

    The seventeen earlier fixes: verified, not assumed

    I re-ran every original repro. All landed. Several exceed what was asked:

    Highlights from the rc.2/rc.3 window

    One operational note, not a defect

    On this build /api/v1/meta/object takes ~11s (680 KB) and /meta/view ~9s. The Console blocks its splash on both, so a cold /_console/ load sits on "Connecting to data source" for ~20s before rendering. It does render correctly. Worth a look before GA if that is not expected for a 24-object app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

domain:cliRetired 2026-10-22, alias of domain:server (cli surface); in-flight cards keep it until they landtarget:v17Release blocker for v17 — on the board until fixed, dropped as no longer valid, or accepted for GAtrackingGenerated anchor/tracker issue — not dispatchable; exempt from the half-state sweep's findings

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions