You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
is 34 minutes long enough to replicate huge files? #49
Low priority - haven't seen this in the wild recently. Guesstimates - based on assuming we can 50% saturate a 1 Gbps connection - are that we can do a 128 GB - about the size of the biggest blu-ray.
Tangential angle from the PR #67 review: the systemd unit (simpler-objects-async-replicate.service) is Type=oneshot with no RuntimeMaxSec=. If the in-code TIMEOUT is ever inadequate — huge file, frozen connection, stuck disk on one storage node, unresponsive locator — a hung run will sit in the active queue and back-pressure the timer (the next tick reuses the oneshot's queue).
Worth pairing whatever value we pick here with RuntimeMaxSec= on the unit, set slightly above the in-code ceiling, as a belt-and-braces kill. That way any code path that bypasses the in-code timeout (transport-level, system-level) still gets bounded.
$ simpler-objects-async-replicate --replicas 2 http://localhost:29164/ bucket
Traceback (most recent call last):
File "simpler-objects-async-replicate", line 8, in <module>
sys.exit(cli())
~~~^^
File "simpler_objects/async_replicate.py", line 123, in cli
results = [auto_replica(args.locator, b, args.replicas) for b in buckets]
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "simpler_objects/async_replicate.py", line 83, in auto_replica
res.raise_for_status()
~~~~~~~~~~~~~~~~~~~~^^
File "httpx/_models.py", line 829, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '503 Service Unavailable' for url 'http://localhost:29164/bkup-20250818/'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/503
There is an 8 second timeout for the locator, but the locator API only waits 2 seconds for the object servers. If ANY timeout it will 503 back. 2 seconds is usually enough unless an HDD is in powersave mode and has to spin up.
The locator 503 is indeed separate but we need to track it here as part of locator (and probably clients generally) timeouts. If we increase the locator timeout then the replicator timeout needs to be greater or equal.
Activity
ctengel commentedon May 23, 2026
Low priority - haven't seen this in the wild recently. Guesstimates - based on assuming we can 50% saturate a 1 Gbps connection - are that we can do a 128 GB - about the size of the biggest blu-ray.
ctengel commentedon May 27, 2026
Tangential angle from the PR #67 review: the systemd unit (
simpler-objects-async-replicate.service) isType=oneshotwith noRuntimeMaxSec=. If the in-codeTIMEOUTis ever inadequate — huge file, frozen connection, stuck disk on one storage node, unresponsive locator — a hung run will sit in the active queue and back-pressure the timer (the next tick reuses the oneshot's queue).Worth pairing whatever value we pick here with
RuntimeMaxSec=on the unit, set slightly above the in-code ceiling, as a belt-and-braces kill. That way any code path that bypasses the in-code timeout (transport-level, system-level) still gets bounded.ctengel commentedon May 28, 2026
maybe related
There is an 8 second timeout for the locator, but the locator API only waits 2 seconds for the object servers. If ANY timeout it will 503 back. 2 seconds is usually enough unless an HDD is in powersave mode and has to spin up.
Two non-mutually-exclusive options
ctengel commentedon May 28, 2026
The locator 503 is indeed separate but we need to track it here as part of locator (and probably clients generally) timeouts. If we increase the locator timeout then the replicator timeout needs to be greater or equal.