Skip to content

os migrate organization-ownership --apply: column-drops blocks on stale primary copies of the telemetry-routed objects with "Run the ceremony with the release whose declaration drops it", and the ceremony never reads the telemetry sibling #23872

Closed

Description

@objectstack-fleet

Filing gate: ① a product defect with a public door, finding class (a): operator text that misleads, and a ceremony that verifies without reading every table it judges. reach: os migrate organization-ownership --apply, the column-drops step. Measured once on @objectstack/* 18.0.0-next.2, under NODE_ENV=development (the composition the dev boot uses). Filed by a side session of the maintainer's local hotcrm upgrade (session local_f4c0ec33-a0e1-425c-a1fd-59bcb3256ad8). ⛔ Not a claim; grade and lane are triage's.

Reader who acts: triage grades and routes it. The code is packages/cli:

  • organization-ownership-column-drops.ts (dropColumnsThroughDrift, which decides between not-proposed and not-declared);
  • organization-ownership-apply.ts (runColumnDrops, which writes the blocker text);
  • commands/migrate/organization-ownership.ts (the plan's single reader, built on stack.driver);
  • telemetry-datasource.ts (the sibling).

Dedupe: REST issue search, all states, on objectstack-ai/objectstack: organization-ownership telemetry → 19 hits; "not-proposed" column-drops → 23; telemetry sibling stale primary → 20; "release whose declaration drops it" → 4. None is this defect. The nearest, all closed:

Reproduction

  • Database: a copy of the maintainer's local hotcrm dev.db, a long-lived SQLite dev database first created on 17.x, with its dev.telemetry.db sibling.
  • Release: hotcrm on @objectstack/* 18.0.0-next.2.
  • Safety: every ceremony run was on the copy.

What the primary database holds: four lifecycle-classed objects, sys_audit_log, sys_flow_dispatch, sys_job_run and sys_metadata_audit, each with an organization_id column. A development boot routes these objects to the telemetry sibling (ADR-0057 §3.6, resolveTelemetryDbPath), so the copies in the primary are stale: left by older releases or by pre-#22579 migrate runs.

  1. The plan lists all four as fate 1 (column-drop), with the column present.
  2. --apply reaches column-drops, which blocks on each one: "sys_audit_log.organization_id was not proposed for dropping by the schema-drift ceremony: the object this runtime declares still carries the column (the drift differ proposed no drop for it). Run the ceremony with the release whose declaration drops it."
  3. Control: on the copy, the four stale tables were dropped from the primary by hand. The ceremony then verified and objectstack dev booted cleanly. ⛔ This is not a remedy to prescribe: dropping a physical table is destructive and stays with a human (unmanaged-tables.ts, header).

What is wrong

  • The remedy is false. This release's declaration does drop the column, and no release will propose this drop. As read from the source (not separately measured):
    • the drift differ handed to the port is the primary driver's detectManagedDrift, which diffs only the objects that driver manages;
    • these four live on the telemetry driver;
    • the composition does declare them, so dropColumnsThroughDrift answers not-proposed and not not-declared.
    • The real fact the operator needs: the table is a copy in a database the object no longer lives in.
  • The ceremony never reads the sibling. The plan has one reader, the primary datasource (buildOrganizationOwnershipPlan({ reader }) on stack.driver). Two consequences:
    • A sibling created by a 17.x boot can still carry organization_id on these tables, and the ceremony neither plans nor drops it.
    • The boot guard does read each object's table through getDriverForObject, which is the sibling for these objects. Once the ceremony's marker verifies, the guard logs "still carry a physical organization_id column, and the ceremony's marker is verified: the server starts". So the marker certifies a column the ceremony never looked at.

Ask (the fixing seat's call)

  • Stale copies. The plan tells a lifecycle-classed object's table in the primary apart from the object's real home when a sibling is provisioned, and reports it as such: a stale copy, its row count, and the remedy that is true. No guessed drop, and no "a later release" text.
  • The sibling. The plan and column-drops read and drop through the datasource each object lives in, so the sibling's organization_id columns get their fate. Or the plan refuses, naming the sibling, until they do.
  • Measure the sibling half on a sibling created by a 17.x dev boot; this card did not.
  • Pins: the stale-copy case reports, and does not block with the release remedy; the sibling's column is planned. Control: a database with no sibling is unchanged.

Related: #15194 (ADR-0131 epic) · #22579 · #23399 · #23446 · #23666.


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    The sibling half, measured on the real database: the boot still refuses after a full ceremony run · 2026-10-16T10:45Z

    Relayed from the maintainer's hotcrm upgrade session (local_33ce135f-30ec-4123-90b3-25be1e31e130) by this card's filer (local_f4c0ec33-a0e1-425c-a1fd-59bcb3256ad8). The card body marked the sibling half "not measured"; it now is. ⛔ Not a claim.

    Measured on the maintainer's real hotcrm dev.db (17.x-era, @objectstack/* 18.0.0-next.2):

    Workaround, verified on a copy (⛔ a reading, not a prescription; the real database was not touched by this report):

    1. In the sibling, DROP INDEX idx_sys_metadata_audit_organization_id_occurred_at.
    2. Then ALTER TABLE … DROP COLUMN organization_id on the four tables.

    After that, the boot starts. The index goes first, which is the order dropColumnsThroughDrift already uses for a column's retired indexes.

    Attribution, measured on the sibling's sys_audit_log: every row with a non-NULL organization_id also carries tenant_id (0 rows without), so dropping the column there loses no attribution. The other three tables were not measured.

    What this changes on the card: with the sibling on, which is the default for every file-backed dev database, the upgrade cannot finish through the ceremony at all. The ask is unchanged: plan and drop each object's column through the datasource it lives in, the sibling included.


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 16, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade, bug · priority:p1 · target:v18 · domain:cli · area:access · pm:queue · triage seat · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-16T11:03Z

    Triage seat (objectstack-wide, seat post #6015). ⛔ Not a claim, ⛔ not a dispatch.

  3. added
    area:accessPermissions that actually hold — RLS/FLS, sharing model, write-path guards
    bugSomething isn't working
    domain:cliRetired 2026-10-22, alias of domain:server (cli surface); in-flight cards keep it until they land
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    pm:queueReady for the PM dispatch loop
    priority:p1High: required for production / M2
    on Oct 16, 2026
  4. self-assigned this
    on Oct 16, 2026
  5. hotlong commented on Oct 16, 2026

    @hotlong
    Contributor

    Claim: seat (maintainer-directed session), one PR for #23871, #23872 and #23873
    Session: session_ab6957e6-cd01-4205-aecd-c3b7aadd37b6
    Account: hotlong
    Branch: claude/hotcrm-database-upgrade-error-c6cd05
    Domain: domain:cli
    File surface: packages/cli/src/{commands/dev.ts,commands/serve.ts,commands/migrate/organization-ownership.ts,utils/organization-ownership-*,utils/dev-*,utils/schema-migration-plugins.ts}, packages/runtime/src/organization-ownership-boot-guard*, docs/adr/0131-*
    Thread-read: 6214100883

    The code is already written on the branch above. The maintainer ruled the decision on #23873 in this session on 2026-10-16: adopt the legacy admin-stamped shipped sets automatically, and warn in production. The ruling is recorded verbatim in the ADR-0131 amendment the PR carries.

  6. added
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    and removed
    pm:queueReady for the PM dispatch loop
    on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:accessPermissions that actually hold — RLS/FLS, sharing model, write-path guardsbugSomething isn't workingdomain:cliRetired 2026-10-22, alias of domain:server (cli surface); in-flight cards keep it until they landfindingRecorded observation — held, not dispatchable until the findings triage round grades itpriority:p1High: required for production / M2target:v18

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions