Repository navigation
Client.compute doesn't materialize delayed chunk of persisted array if provided as a dict value #8959
Description
Describe the issue:
When persisting a dask array and trying to compute one of its chunks, the return value is the dask key instead of the computed array if the chunk is provided as a dict value.
Minimal Complete Verifiable Example:
import dask.array as da
from dask.distributed import Client
if __name__ == '__main__':
client = Client()
X = da.random.random((100, 2), chunks=(100, 2)).persist()
delayed_chunk = X.to_delayed()[0, 0]
print(type(client.compute(delayed_chunk).result())) # numpy.ndarray
print(type(client.compute([delayed_chunk])[0].result())) # numpy.ndarray
print(type(client.compute({'x': delayed_chunk}).result()['x'])) # tuple (the dask key)Anything else we need to know?:
On 2024.11.2 these all return an array.
Environment:
- Dask version: 2024.12.0
- Python version: 3.10.16
- Operating System: Ubuntu
- Install method (conda, pip, source):
pip install dask[distributed]
Reacted by Christian Lorentzen
Reacted by James Lamb
Activity
[-]`Client.compute` doesn't materialize delayed array chunk of persisted array if provided as a dict value[/-][+]`Client.compute` doesn't materialize delayed chunk of persisted array if provided as a dict value[/+]jrbourbeau commentedon Dec 9, 2024
Thanks for the issue and nice minimal example @jmoralez -- I'm able to reproduce. @hendrikmakait maybe this is similar to some of the other
client.computetask spec stuff you did a week or two ago?IzerOnadimQC commentedon Dec 10, 2024
The breakage is not coming from
distributedbutdaskitself.I think the responsible commit is dask/dask@5b115c4. I can reproduce the behaviour described above when installing
distributed 2024.12.0along with that particular commit, but installing the previous commit (dask/dask@fe9f453) restores the old behaviour.@phofl I don't understand these changes well enough to see the issue. Could you please take a look?
jameslamb commentedon Mar 24, 2026
I think this may have been fixed.
@jmoralez originally found it in LightGBM (lightgbm-org/LightGBM#6739) and I accidentally ran LightGBM's tests in an environment with
dask/distributed2026.1.0 and saw them all pass.I did a binary search over
daskversions in my local environment, using the reproducible example from this issue.It looks like this was fixed in
2025.4.0.There weren't many commits in 2025.3.0...2025.4.0, so hopefully maintainers could figure out which one fixed this and turn that into a regression test here.
Or just close this if you agree it's fixed, whatever you all want.
Thanks for the help!