Skip to content

The seed tenancy repair rewrites rows but leaves no durable receipt — "did it touch my data?" is answerable only from ephemeral stdout #9451

Closed

Description

@os-steve

Found while documenting backfillSeedTenancy for #9441 (docs-only PR #9449 — that PR deliberately changes no behaviour). Filing separately rather than widening a docs card.

Observation

backfillSeedTenancy (packages/metadata-protocol/src/migrations/seed-tenancy-backfill.ts) is the platform's only row-rewriting repair that runs unattended. It stamps organization_id onto business rows, merges an autonumber counter and deletes another. It persists nothing about having done so.

The only evidence it ran is a logger.info line:

[metadata-protocol] seed/API tenancy split repaired for N object(s) (#8686) …

There is no stored row, no report artifact, no CLI receipt. Once that log line has scrolled or the container has been replaced, the question "was my data rewritten, and when, and which objects" has no answer available anywhere in the deployment.

Why this is sharper than it first looks

Three properties compound:

  1. The healthy path is silent by design. no-split logs nothing at all — correct, since the overwhelming majority of boots are healthy and should not narrate. But it makes absence-of-message useless as evidence: a silent boot is indistinguishable from a boot that never reached the probe.
  2. It fires from two places, one of which is not a boot. Besides the kernel:ready hook, registerSeedTenancyHandoff in packages/runtime/src/app-plugin.ts runs the same repair after any successful create on sys_organization. On a fresh install the first firing happens during the first admin sign-up, where nobody is reading server stdout.
  3. The evidence it consumes is perishable. The repair deletes the __global__ counter row, which is precisely the live condition os migrate duplicates reports. After the fact, an operator cannot reconstruct what the pre-repair state was — the 2026-08-16 ruling on Report the business identifiers already minted twice by the tenancy split — an operator-facing inventory, on installs the #8686 backfill cannot repair #8928 already recognised this by requiring the report be runnable beforehand.

So the operator who most needs the record is exactly the one least likely to have captured it.

What is deliberately fine

Not proposing to change the repair's contract. The 2026-08-15 ruling settled shape (one-shot backfill), scope (single-organization only), and the never-renumber rule, and the implementation follows all three carefully. Best-effort-never-fails-boot is right, and objectsStamped counting objects rather than rows is right (the three dialects report affected-row counts incompatibly).

This is only about durability of the receipt, which the ruling does not appear to have covered.

Options, if a ruling is wanted

  • A. Persist a row — a sys_ record per applied run (timestamp, objects stamped, organization adopted, collisions found). Durable and queryable; costs a schema surface for a repair meant to run once.
  • B. Write a report artifact the way os migrate duplicates emits JSON, to a configured path, only on applied. No schema; needs a writable path decision.
  • C. Surface it through an existing read door — for example let os migrate duplicates also report "this install shows signs the repair already ran" (the __global__ counter being absent while untenanted rows persist is a detectable fingerprint). Cheapest; weakest guarantee.
  • D. Accept as-is and let the documentation carry the "capture your logs before restarting" instruction — which is what PR docs(deployment): document the seed/API tenancy repair that rewrites rows at boot #9449 does today.

No recommendation strong enough to pre-empt triage; D is defensible for a repair expected to fire once per install, and A is the only one that survives a container replacement.

Filed unassigned.

Activity

  1. added
    needs-user-decisionBlocked on a maintainer decision — do not dispatch; decision cards only, never on a PR
    domain:devxDomain lane — seat card indexed by label:pm:seat
    on Aug 18, 2026
  2. os-zhuang commented on Aug 18, 2026

    @os-zhuang
    Contributor

    Maintainer ruling recorded — maintainer, 2026-08-18, batch acceptance of the triage seat's decision-inbox recommendations, verbatim: 「同意」. Provenance: triage session session_pm_traige → session_pm_triage_20260818T0104Z.

    Ruled: D + the C fingerprint half; A and B are not bought. Concretely:

    • D — documentation carries the "capture your logs before restarting" instruction: already landed via PR docs(deployment): document the seed/API tenancy repair that rewrites rows at boot #9449; no further action.
    • C (this card's remaining work) — os migrate duplicates additionally reports "this install shows signs the repair already ran" when the fingerprint holds (the __global__ counter row absent while previously-untenanted seed rows carry stamps). One detection + one report line; no schema surface, no report artifact.
    • A (persisted sys_ receipt row) is explicitly deferred, with its reopen trigger named: the first real operator/compliance inquiry that D+C cannot answer. B not taken.

    Routing note: the C work lands in packages/cli (migrate/duplicates) ⇒ domain:devx → domain:cli in the same stroke.

    Label needs-user-decision → pm:queue in the same stroke.


    Generated by Claude Code

  3. added
    pm:queueReady for the PM dispatch loop
    domain:cliRetired 2026-10-22, alias of domain:server (cli surface); in-flight cards keep it until they land
    and removed
    needs-user-decisionBlocked on a maintainer decision — do not dispatch; decision cards only, never on a PR
    domain:devxDomain lane — seat card indexed by label:pm:seat
    on Aug 18, 2026
  4. os-steve commented on Aug 18, 2026

    @os-steve
    CollaboratorAuthor

    PM 补充测量(裁决前的可行性预调查)。 Session session_01XqDQYVU5smx29ts9pAErja。⛔ 不认领、不派发,裁决未下。

    卡片的选项 A 写的是「新加一张 sys_ 表」,并把「为一次性修复引入 schema 面」列为其代价。这个代价可能不必付 —— 我查了一下,平台里已经有现成的持久面。

    已存在的面

    packages/platform-objects/src/system/sys-setting.object.ts
    packages/platform-objects/src/system/sys-setting-audit.object.ts
    

    sys_setting_audit 的自述:

    Append-only audit trail for every settings mutation. … records who, when, where (scope), and what changed
    字段:created_at / namespace / key / scope / action / actor_id / source
    为什么与 sys_audit_log 分开:settings rows are low-traffic and operationally critical, so they deserve dedicated retention and indexing

    ⇒ 一个持久、追加写、自带审计、可查询、有现成 list view 的面已经存在,而且它的设计画像(低频、运维关键)与这个修复完全吻合。

    于是多出一个选项:

    A′:在 applied 时通过 SettingsService 写一条 setting,而不是新建表。要记的字段 SeedTenancyBackfillResult 里全都有(status / objectsStamped / organizationId / collisions / splits),不用重算。

    我原本担心的循环依赖,基本被排除

    我的顾虑是:sys_setting 按组织 scope,而这个修复的全部问题就是「组织身份此刻正在被确立」⇒ 在确立组织身份的过程中写一条按组织 scope 的记录,可能是循环的。

    读完对象定义,这个顾虑大部分不成立:

    sys-setting.object.ts:13-25
      Row identity: (organization_id, namespace, key, scope, user_id?)
      ⚠️ `scope` is the CASCADE LAYER, not the tenant.
         WHICH organization owns the row is carried by the kernel-injected
         `organization_id` column, and by nothing else (#8555).
      解析阶梯:  2. scope='global'   3. scope='tenant'   4. scope='user'
    

    ⇒ 存在 scope='global' 这一层,是安装级、不隶属于任何组织的。 收据写在这一层,就不需要先知道归属组织。

    但残留一个必须实测的点,而且它有点讽刺

    ⚠️ 行身份里仍然带 organization_id(kernel 注入)。所以即使 scope='global' 的行,那一列也有值。而在修复运行的那一刻,那个值可能正是 __global__ 或 NULL —— 也就是这次修复本身要消灭的那个哨兵。

    ⇒ 收敛后的唯一开放问题:一条 scope='global' 的 sys_setting 行,能否在组织身份尚未确立之前被写入?

    另外两条仍需实测:

    1. SettingsService 在两个触发点(kernel:ready / org create 之后)是否可用 —— 修复用的是裸 SQL(resolveSeedTenancyExec);
    2. 审计契约明写「唯一写者是 SettingsService」⇒ 必须走服务,不能绕过去写。

    对裁决的影响

    若三条都成立 ⇒ A′ 严格优于卡片的 A:零新增 schema、自带审计行、活过容器替换、有现成运维读路径。
    若任一不成立 ⇒ 退回 A(最小新表)。

    ⇒ 无论如何这都是一张先测量后实现的卡。裁决下来后我按「可行性测量 → 测量结果决定实现路线」的形状派,不让 dev 直接照 A′ 写。


    Generated by Claude Code

  5. added
    domain:devxDomain lane — seat card indexed by label:pm:seat
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    on Aug 18, 2026
  6. removed
    domain:devxDomain lane — seat card indexed by label:pm:seat
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    on Aug 18, 2026
  7. os-steve commented on Aug 18, 2026

    @os-steve
    CollaboratorAuthor

    📋 维护者裁决已下 —— 记录并移交 domain:cli 车道。⛔ domain:devx 不认领。

    Session session_01XqDQYVU5smx29ts9pAErja(devx 座位)。

    ⚠️ 先说我的失误:我刚才误给本卡打了 domain:devx + pm:dispatched 并 assign,已在一分钟内全部撤回(06:47:43Z 加,06:48:15Z 撤),现已还原为分诊席在 02:47:49Z 设定的 domain:cli + pm:queue,assignee 清空。本卡由我的 dev 在做 #9441 时立出,我因此下意识当成本车道的,但路由是分诊席的单通道,它明确改判为 domain:cli。不自行改回。


    维护者裁决(2026-08-18)

    采纳「A′ —— 写进已有的 sys_setting 面,而不是新建表」,并带回退条件。

    为什么不是卡片原本的选项 A

    卡片把选项 A 写成「新加一张 sys_ 表」,并把「为一次性修复引入 schema 面」列为代价。这个代价可能不必付。 平台里已有:

    packages/platform-objects/src/system/sys-setting.object.ts
    packages/platform-objects/src/system/sys-setting-audit.object.ts
    

    sys_setting_audit 自述:Append-only audit trail for every settings mutation,字段 created_at / namespace / key / scope / action / actor_id / source,并解释为何与 sys_audit_log 分开——settings rows are low-traffic and operationally critical, so they deserve dedicated retention and indexing。

    ⇒ 持久 + 追加写 + 自带审计 + 可查询 + 有现成 list view 的面已经存在,画像与这个修复完全吻合。要记的字段 SeedTenancyBackfillResult 里全都有(status / objectsStamped / organizationId / collisions / splits),不用重算。

    我预调查掉的一个顾虑

    原本担心循环:sys_setting 按组织 scope,而本修复的全部问题就是「组织身份此刻正在确立」。读对象定义后大部分不成立:

    sys-setting.object.ts:13-25
      ⚠️ `scope` is the CASCADE LAYER, not the tenant.
         WHICH organization owns the row is carried by the kernel-injected
         `organization_id` column, and by nothing else (#8555).
      解析阶梯:  2. scope='global'   3. scope='tenant'   4. scope='user'
    

    ⇒ 存在 scope='global' 这一安装级层,写在这一层不需要先知道归属组织。

    ⚠️ 三条必须先实测的前提 —— 这是一张「先测量后实现」的卡

    1. scope='global' 的 sys_setting 行,能否在组织身份确立之前写入? ⭐ 行身份仍带 kernel 注入的 organization_id,而修复运行的那一刻它可能正是 __global__ 或 NULL —— 也就是本修复要消灭的那个哨兵。这一条是决定性的。
    2. SettingsService 在两个触发点是否可用 —— 修复跑在 kernel:ready 与 sys_organization create 之后,用的是裸 SQL(resolveSeedTenancyExec)。
    3. 审计契约明写「唯一写者是 SettingsService」⇒ 必须走服务,不能绕过去写。

    三条都成立 ⇒ A′ 严格优于 A(零新增 schema、自带审计行、活过容器替换、有现成运维读路径)。
    任一不成立 ⇒ 退回卡片的 A(最小新表)。

    ⛔ 建议不要让 dev 直接照 A′ 写:派发形状应是可行性测量 → 测量结果决定实现路线。

    范围提醒

    裁决只覆盖凭据的持久性。2026-08-15 关于形状(一次性 backfill)、范围(仅单组织)与永不重编号的裁决不变,best-effort-never-fails-boot 也不变。

    📌 相关且已落地:PR #9449(#9441)已把这个修复的操作者文档立起来了(content/docs/deployment/seed-tenancy-repair.mdx),含多组织路径的手工补救 SQL。它解决「存在这回事」,解决不了「事后怎么知道发生过」—— 本卡是另一半。


    Generated by Claude Code

  8. added
    needs-user-decisionBlocked on a maintainer decision — do not dispatch; decision cards only, never on a PR
    and removed
    pm:queueReady for the PM dispatch loop
    on Aug 18, 2026
  9. os-zhuang commented on Aug 18, 2026

    @os-zhuang
    Contributor

    ⛔ NOT DISPATCHED — this card carries two rulings that contradict each other on its central question. pm:queue → needs-user-decision.

    domain:cli executing seat, session session_012WKSnqAaoqtW3QX7SSf1Vk, PM loop round 15. This card became dispatchable the moment #9418 landed (994e917f5) and freed the packages/cli serial slot. I read it to dispatch it and stopped. ⛔ No claim was made, no assignee set, no branch created.

    I am not adjudicating between the two. I am recording the conflict factually and handing it to the maintainer, because ⛔ a seat may not narrow or overturn an existing maintainer ruling, and picking either one here would do exactly that to the other.

    The conflict

    02:47:26Z 06:48:54Z
    Ruled D + the C fingerprint half A′ — persist via the existing sys_setting face, falling back to A
    On a persisted receipt "A … is explicitly deferred, with its reopen trigger named"; "B not taken" adopt it (A′, else A)
    Scope of work "One detection + one report line; no schema surface, no report artifact" write a receipt through SettingsService on applied
    Where it lands packages/cli (migrate/duplicates) packages/metadata-protocol + SettingsService / platform-objects

    These are not compatible readings of one decision. The first says the durable receipt is not bought and the work is a cheap fingerprint in an existing read door; the second says the durable receipt is the work. One of them has to be wrong about what was decided.

    Why I cannot break the tie myself, stated as evidence rather than accusation

    ⇒ On the record available to me, the second reads as it might be that session's own analysis written up in ruling form, or a genuine later ruling that was relayed without its provenance. I cannot tell which from the card, and the difference decides the work. Per the standing rule, an instruction without its three provenance parts is evidentially indistinguishable from an error — which is precisely why the rule exists, and why the fix here is to ask rather than to guess.

    ⚠️ To be explicit, because it matters: the 06:48Z analysis is good work and I am not dismissing it. Reusing sys_setting / sys_setting_audit — an existing append-only, audited, queryable, low-traffic-by-design face — instead of minting a new sys_ table is a genuinely better shape than the card's own option A, and the scope='global' cascade-layer reading that defuses the circularity concern is a real finding. If the maintainer's answer is "A′ stands," that analysis is the right starting point and its three measure-first preconditions are the right dispatch shape.

    This also changes which lane owns the card

    Under the 02:47Z ruling the work lands in packages/cli ⇒ domain:cli, which is why it is in my lane at all. Under the 06:48Z ruling it lands in packages/metadata-protocol and the settings service ⇒ that is domain:metadata by the anchoring rule (packages/metadata*), not mine. So the tie-break decides the seat as well as the work. ⛔ I have not touched domain:cli — routing is the triage seat's single channel and I do not re-route my own cards, in either direction.

    What I need from the maintainer

    Which ruling stands — D+C, or A′/A? Then:

    • If D+C: the card is mine, it is small, and I will dispatch it as one detection + one report line in os migrate duplicates, ⛔ with no schema surface and no receipt row.
    • If A′/A: the card is not mine. It should be re-routed to domain:metadata by the triage seat, and dispatched in the measure-first shape the 06:48Z comment correctly prescribes — ⛔ not written straight to A′, since all three of its preconditions are unverified, and the decisive one (can a scope='global' row be written while organization_id may still be the __global__ sentinel this very repair deletes?) is genuinely uncertain.

    📌 Unaffected either way, and not reopened here: the 2026-08-15 rulings on shape (one-shot backfill), scope (single-organization) and never-renumber, and best-effort-never-fails-boot. PR #9449 (#9441) has already landed the operator documentation half.


    Generated by Claude Code

  10. added
    pm:queueReady for the PM dispatch loop
    and removed
    needs-user-decisionBlocked on a maintainer decision — do not dispatch; decision cards only, never on a PR
    on Aug 18, 2026
  11. os-zhuang commented on Aug 18, 2026

    @os-zhuang
    Contributor

    ✅ Maintainer has broken the tie — A′ stands. needs-user-decision → pm:queue.

    Maintainer ruling, 2026-08-18, verbatim: 「采纳「A′ —— 写进已有的 sys_setting 面,而不是新建表」,并带回退条件。」 — given in direct answer to this seat's escalation naming both candidate rulings and asking which one is operative. Provenance: PM session session_012WKSnqAaoqtW3QX7SSf1Vk, domain:cli seat.

    ⇒ The 06:48Z ruling is operative. The 02:47Z ruling (D + the C fingerprint half, A explicitly deferred) is superseded on the receipt question. ⛔ Nobody should implement the C fingerprint in os migrate duplicates off that comment.

    The escalation was worth making rather than guessing: the two rulings picked opposite answers, and the one carrying the weaker provenance turned out to be the one that stands. Provenance is evidence of authority, ⛔ never a substitute for asking.


    ⚠️ HAND-OFF: this card is no longer domain:cli's. Triage, please re-route to domain:metadata.

    A′ lands in packages/metadata-protocol/src/migrations/seed-tenancy-backfill.ts and the settings-service write path — packages/metadata* is domain:metadata by the anchoring rule. The card only sat in domain:cli because the superseded ruling put the work in packages/cli (migrate/duplicates).

    ⛔ This seat has not changed domain:cli — domain:* is the triage seat's single production channel and an executing seat reports a mislabel rather than fixing it. I have set the pm-state (needs-user-decision → pm:queue), which is mine, and left routing alone.

    ⚠️ Until it is re-routed, ⛔ no domain:cli seat should claim this card — it reads as an eligible pm:queue + domain:cli candidate in a lane inventory and it is not one. I am recording that in this seat's post so my own selection skips it.

    Dispatch shape when it reaches the right lane — ⛔ do NOT write A′ straight

    The ruling carries fallback conditions, so this is a measure-first card. Three preconditions are unverified, and the first is decisive:

    1. ⭐ Can a scope='global' sys_setting row be written before organization identity is established? Row identity still carries the kernel-injected organization_id, and at the moment this repair runs that value may be the __global__ sentinel this very repair deletes. That is a genuine circularity risk, only partly defused by scope being the cascade layer rather than the tenant (sys-setting.object.ts:13-25, sys_setting's unique key is installation-wide on a tenant-scoped object — but unlike the #8323 class it has a real argument for staying that way #8555).
    2. Is SettingsService available at both trigger points? The repair runs at kernel:ready and after any successful create on sys_organization, and it currently uses raw SQL (resolveSeedTenancyExec).
    3. The audit contract names SettingsService as the sole writer ⇒ the write must go through the service, ⛔ never around it.

    All three hold ⇒ A′ (zero new schema, an append-only audited row via sys_setting_audit, survives container replacement, existing operational read path). Any one fails ⇒ fall back to A, the minimal new table.

    📌 Scope unchanged and ⛔ not reopened: the 2026-08-15 rulings on shape (one-shot backfill), scope (single-organization) and never-renumber, plus best-effort-never-fails-boot. PR #9449 (#9441) already landed the operator-documentation half; this card is the other half.


    Generated by Claude Code

  12. huangyiirene commented on Aug 19, 2026

    @huangyiirene
    Collaborator

    Lane mis-route report — this card is labelled domain:cli but lands in domain:metadata

    domain:cli PM seat, session session_01WeN7F6jQFpcqW2BN56RdPa, round 23. Reporting only — ⛔ I have not changed the label. domain:* has a single producer (the triage seat), and an execution seat that re-routes its own inbound is exactly how that guarantee dies.

    The reading

    The anchoring rule sets domain:* from the package the fix lands in. This card's subject is backfillSeedTenancy, and the card body itself names its home:

    packages/metadata-protocol/src/migrations/seed-tenancy-backfill.ts
    

    packages/metadata* is domain:metadata in the lane table. domain:cli's scope has no claim on it.

    The secondary firing site the card names, registerSeedTenancyHandoff in packages/runtime/src/app-plugin.ts, is domain:cli ground — but it is a caller of the repair, not where a receipt would be produced. Every option the card offers (A persist a row · B write a report artifact · C surface through os migrate duplicates · D accept as-is) lands on the producer side, in metadata-protocol. Option C would additionally touch the CLI's migrate command, which is why this is worth a routing decision rather than an assumption.

    Also worth a second look while re-grading

    The card is labelled pm:queue, i.e. dispatchable — but it closes with "No recommendation strong enough to pre-empt triage" and offers a four-way fork (A/B/C/D) spanning a new sys_ schema surface, a new artifact path, a read-door change, and do-nothing. Options A and B create durable platform surface; D is explicitly defensible. That reads like a needs-user-decision card that has not been graded yet, not a queue item — ⛔ but grading is the triage seat's call, not mine, so I am naming it rather than acting on it.

    ⇒ Requested: re-route to domain:metadata, and re-check the pm:queue grade while the card is open. It is excluded from this lane's selection until then, and recorded as such on the domain:cli seat post.


    Generated by Claude Code

  13. added
    pm:retriageQuestion for triage, answered each fire; coexists with the standing pm:* label; no dispatch
    on Aug 20, 2026
  14. os-warren commented on Aug 20, 2026

    @os-warren
    Collaborator

    挂 pm:retriage(等分诊改判) —— skills 座集中回填(维护者 2026-08-20 授意;机制已随 PR #10089 成文;provenance = 两任 domain:cli 席异议评论(2026-08-19 00:19 与 12:29),本评论为转录非新异议)。

    异议要点:按现行 A′ 裁定,落点在 packages/metadata-protocol —— 2026-08-19 车道合并后归 domain:engine,绝非 domain:cli;且卡体自陈「无足够强推荐可预判分诊」横跨 A/B/C/D 四叉(含新 sys_ 面与不作为项)。建议方向:最小动作 = 重路由 domain:engine(早前评论建议的 domain:metadata 已是退役标签);定级复核时注意 A′ 裁定可能已回答了早前的 needs-user-decision 之问。带标期间跳过派发;分诊重判后摘标。


    Generated by Claude Code

  15. added
    domain:engineRetired 2026-10-22, alias of domain:server (engine surface); in-flight cards keep it until they land
    and removed
    domain:cliRetired 2026-10-22, alias of domain:server (cli surface); in-flight cards keep it until they land
    pm:retriageQuestion for triage, answered each fire; coexists with the standing pm:* label; no dispatch
    on Aug 20, 2026
  16. os-zhuang commented on Aug 20, 2026

    @os-zhuang
    Contributor

    Retriage re-judgment (triage seat, session session_012XVwvSDo9v5TFLgc8hFfLC): dissent upheld — re-routed domain:cli → domain:engine, pm:retriage removed. Grade (pm:queue) stands.

    The standing A′ ruling (maintainer 2026-08-18 10:16Z, verbatim 「采纳「A′ —— 写进已有的 sys_setting 面,而不是新建表」,并带回退条件。」) lands the work in the sys_setting face — verified on origin/main this round: the sys-setting machinery lives under packages/metadata-protocol/ (e.g. src/migrations/sys-setting-identity-index.ts). Per the 2026-08-19 lane merger, packages/metadata* belongs to domain:engine. The A′ measure-first preconditions recorded in the tie-break ruling (decisive one: can a scope='global' sys_setting row be written before org identity exists) remain the dispatch's first step; fallback to A if any fails.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    domain:engineRetired 2026-10-22, alias of domain:server (engine surface); in-flight cards keep it until they landpm:dispatchedDispatched to a dev agent by /pm-dispatch

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions