Async replication #5

Activity

1. ctengel

   added this to the Basic milestone on Aug 17, 2025
2. ctengel

   added a commit that references this issue on Aug 17, 2025

   Basic bucket listing and replication

   ...

   d765209
3. ctengel

   self-assigned this

   on Oct 25, 2025
4. ctengel commented on Oct 25, 2025

   

   ctengel

   on Oct 25, 2025

   OwnerAuthor

   More actions

   Pin
5. ctengel

   pinned this issue on Jan 7, 2026
6. ctengel commented on Jan 7, 2026

   

   ctengel

   on Jan 7, 2026

   Last edited by ctengel

   OwnerAuthor

   More actions

   Next steps

   • Deploy latest code
   • Manually scrub everything
   • Test manual async replication

   From there the next part is build out a client/server system that:

   • Takes multiple sources and multiple destinations
   • Allows user to set tgt number of replicas
7. ctengel

   added a commit that references this issue on Jan 10, 2026

   Script to migrate from old to new checksum style

   ...

   6e7582b
8. ctengel commented on Jan 10, 2026

   

   ctengel

   on Jan 10, 2026

   OwnerAuthor

   More actions

   Option A:

   • build out locator API that is ls with list of locations as part of metadata
   • also one for placing replicas, either:
     • expose known servers
     • modified PUT that instead of 409 will redirect to a server that doesn't have it yet
   • script takes a bucket and a min number of replicas
   • call API and if any below minimum, trigger a replication
     • use above replica placement (pull data and figure one out) or just PUT with appropriate switches
     • or user supplied targets

   Option B:

   • no change to locator or object servers
   • replicator takes source servers, dest servers, and min number of replicas
     • source and dest may overlap or even be the same
   • go to each source and list and store files and metadata along with where each was found
     • error if hash doesn't match
   • go through list and check each dest server, if count is too low, pick from remaining servers appropriate place(s) to replicate (similar to current locator put logic)

   Due to lack of locking #18 #25 maybe refuse to replicate if no sha sum?
9. ctengel

   added 2 commits that reference this issue on Jan 14, 2026

   untested replication stuff

   ...

   b54fbef

   make new replicate code CLI

   ...

   c6c80a8
10. ctengel

    closed this as completedin 46af6d8on Jan 16, 2026
11. ctengel

    unpinned this issue on Jan 16, 2026