Repository navigation
Description
tests/test_sync.py::test_sync_timeout has failed twice on main since 09-23. Both failures were on os=macos-latest, py=3.12, deps=optional, and both were Failed: DID NOT RAISE TimeoutError:
- https://github.com/zarr-developers/zarr-python/actions/runs/37915753695 (2026-10-09)
- https://github.com/zarr-developers/zarr-python/actions/runs/40856533446 (2026-10-29)
The test's coroutine sleeps for 20 ms, and the test expects sync(foo(), timeout=0.002) to time out (tests/test_sync.py#L63-L70). sync() submits the coroutine with run_coroutine_threadsafe and then calls concurrent.futures.wait(..., timeout=timeout) (src/zarr/core/sync.py#L147-L151). Suppose the test thread is not running again within about 18 ms of submitting. Then the coroutine has already finished, wait() returns it in finished, and sync() returns normally. On a busy runner, that 18 ms is not guaranteed.
Reproduction. This version is deterministic: it delays the test thread by 50 ms between submitting the coroutine and waiting on it.
import asyncio
import time
import zarr.core.sync as zsync
real_wait = zsync.wait
def late_wait(*args, **kwargs):
time.sleep(0.05) # stands in for the test thread being descheduled
return real_wait(*args, **kwargs)
zsync.wait = late_wait
async def foo() -> None:
await asyncio.sleep(0.02)
zsync.sync(foo(), timeout=0.002)
print("returned without TimeoutError")Without the patch, the failure is rare. The unmodified test body was run on an 8-core Mac with 24 busy loops alongside it. It missed the timeout in 1 of 2,000 iterations in each of two runs, and in 0 of 4,000 in a third. With the machine idle, it missed in 0 of 1,000.
One way to remove the timing dependence is to keep the coroutine unfinished until the test has seen the timeout:
@@ -1,2 +1,3 @@
import asyncio
+import threading
from collections.abc import AsyncGenerator
@@ -63,9 +64,15 @@ def test_sync_raises(sync_loop: asyncio.AbstractEventLoop | None) -> None:
def test_sync_timeout() -> None:
- duration = 0.02
+ release = threading.Event()
async def foo() -> None:
- await asyncio.sleep(duration)
-
- with pytest.raises(asyncio.TimeoutError):
- sync(foo(), timeout=duration / 10)
+ # Stay unfinished until the timeout has been seen, so the result does not depend
+ # on how soon the calling thread is rescheduled after submitting the coroutine.
+ while not release.is_set():
+ await asyncio.sleep(0.001)
+
+ try:
+ with pytest.raises(asyncio.TimeoutError):
+ sync(foo(), timeout=0.002)
+ finally:
+ release.set()With this change, the delayed-wait setup above still raises TimeoutError in 50 of 50 iterations. pytest tests/test_sync.py also passes (19 passed) under the project's pytest configuration.
The open PR #3997 adds test_run_timeout, which uses the same 20 ms sleep with timeout=duration / 10, so it would hit the same race. The future.cancel() added in #4343 does not change this, because it only runs when wait() reports the future as unfinished.
This was investigated and written up with Claude Code, an AI agent. Before filing, the two run logs were re-read, and the reproduction and the patched test were run on macOS arm64 with Python 3.12.12 and zarr at bf0d231.
Activity
sheikhayaan commentedon Oct 31, 2026
I'd like to pick this up. Using a threading.Event to keep the coroutine unfinished until the timeout is seen seems like the most robust fix for the flakiness. I'll apply the patch to test_sync_timeout and open a PR. Should I also update test_run_timeout in PR #3997 here, or handle that separately?