Skip to content

design(atlas): offset-matching algorithm — rsync-style sliding-window matcher for span alignment #6036

Open

Description

@dezren39

Parent: #6028 (delta-only ingestion + offset embeds). Research source: prior-art doc ~/workspace/scratch/atlas-ingestion-brief-2026-11-04/prior-art-research.md (2026-11-04). Follow-up 2 (Cluster 1).

Finding (research, in substance)

Computing "indices 0–16 matched old span A, 17–241 is new, 242+ matched old span B" without aligned chunk boundaries is the rsync matcher (Tridgell & Mackerras, TR-CS-96-05, 1996): slide a window one byte at a time over the new file computing a cheap rolling (Adler-32-derived) checksum; only on a weak hit pay for the strong checksum; emit literal/copy tokens. http://oceanpark.com/webmuseum/rsync.html — That sliding-window match at arbitrary offsets is exactly the mechanism the offset-embed design needs.

Scope

Done looks like

An algorithm design with parameters, failure cases (reflow/re-wrap phantom diffs), and a worked run over Drew's 17–241 example.

Activity

  1. added
    area/infraInfrastructure, deployment, and operations
    kind/designA design proposal (design-architect): multi-step, phased, decision-bearing
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  4. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  5. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

    Triage findings (verified fresh 2026-11-05)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraInfrastructure, deployment, and operationskind/designA design proposal (design-architect): multi-step, phased, decision-bearingtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions