Skip to content

design(atlas): span-table schema — spans as (file_version_id, start, end) with embedded-text snapshots #6035

Open

Description

@dezren39

Parent: #6028 (delta-only ingestion + offset embeds). Research source: prior-art doc ~/workspace/scratch/atlas-ingestion-brief-2026-11-04/prior-art-research.md (2026-11-04). Follow-up 1 (Cluster 1).

Finding (research, in substance)

The offset-embed model makes a file version an ordered list of span references (embed_id, start, end) — the restic/borg content-addressed span-list shape applied to embeddings. A new version then costs only its genuinely new spans plus a manifest row. The span's stored text is what was embedded, and spans are reused only when the text is byte-identical (embeddings are not hashes: the same paragraph embedded with different neighbors is a different vector).

Scope

  • The span table: (file_version_id, start, end) + the span's own embedded-text snapshot + vector + provenance link.
  • How spans and today's chunk store (~107,639 chunks) coexist during migration: dual-read, backfill, or cutover — and what happens to chunk IDs that existing traces/feedback reference.
  • Span context metadata (heading path / section) versions with the reference, not the span (a reused span under a new "Deprecated:" heading must not inherit stale context).

Done looks like

A schema design: tables, keys, migration path from the chunk store, and the invariants (byte-identical reuse only; context versioned per reference).

Activity

  1. added
    area/infraInfrastructure, deployment, and operations
    kind/designA design proposal (design-architect): multi-step, phased, decision-bearing
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 5, 2026
  4. added
    work/assignedQueue worker lifecycle state
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  5. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor

    🤖 Agent note · handed off

  6. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  7. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    task: 10401
    state: handed_off
    lane: code-a (execution lane: opencode)

    Picked up by Lane A shift (attempt 2/6). Revalidated 2026-11-05: issue #6035 is OPEN, no held / user/owned / do-not-merge / do-not-auto-edit labels. Execution lane is opencode — handed off to the opencode claimant (repeat handoff: prior park expired unbound). Parking 1800s for the claimant window.

  8. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · state: handed off

  9. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    Agent note - handed off

  10. 36 remaining items

  11. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 6, 2026
  12. dezren39 commented on Nov 6, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  13. dezren39 commented on Nov 6, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  14. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    queue/claimedQueue worker lifecycle state
    and removed
    queue/claimedQueue worker lifecycle state
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 6, 2026
  15. dezren39 commented on Nov 6, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraInfrastructure, deployment, and operationskind/designA design proposal (design-architect): multi-step, phased, decision-bearingqueue/claimedQueue worker lifecycle statestatus/in-progressQueue worker lifecycle statetask/triageQueue packet kind: verify, reproduce, and route this itemwork/assignedQueue worker lifecycle state

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions