Update open_dataset with a store to produce non-random dask task names #11814

Is your feature request related to a problem?

I have a custom file reader that tries to work around some of the cache usage of #11581 to share the same NetCDF store for all files I want to read and all groups/variables of that file. To do that I create a NetCDF4DataStore and pass it to open_dataset. However, this creates a unique dask task name because xarray tokenize the store object which ends up tokenizing the lock and the manager_id. This means opening the file twice and getting the same variable does not equal the same dask task and it will be loaded twice from disk.

Side hack: My library also uses the dask task name as a shortcut for two dask arrays being equal to avoid rebuilding a calculation. This issue ends up meaning that this shortcut will never fire/succeed.

import numpy as np
import xarray as xr

xr.Dataset({"a": ("x", np.arange(4))}).to_netcdf("test.nc")

# Opened by filename: the same name every time
print(xr.open_dataset("test.nc", chunks={})["a"].data.name)
print(xr.open_dataset("test.nc", chunks={})["a"].data.name)

# Opened from a store: a new name every time
print(xr.open_dataset(xr.backends.NetCDF4DataStore.open("test.nc"), chunks={})["a"].data.name)
print(xr.open_dataset(xr.backends.NetCDF4DataStore.open("test.nc"), chunks={})["a"].data.name)

# Output
# open_dataset-a-fead3117c70e16f1cc81dca5dd152aa9
# open_dataset-a-fead3117c70e16f1cc81dca5dd152aa9
# open_dataset-a-329b5ec6a4be8658705ea24f1b0ef524
# open_dataset-a-5c0d73a6a698ddfbce4f96ee4aba4eb7

Describe the solution you'd like

I think a __dask_tokenize__ method added to some of the store classes would be the "simplest" fix but I could see there being cases where the subclasses are improperly tokenized (not taking into account some arguments/kwargs, etc). When I asked Claude how it would define this given this concern about mis-tokenizing things it suggested adding __dask_tokenize__ to the CachingFileManager as well:

# CachingFileManager: everything it pickles except the random manager id and the lock,
# plus the mtime, like open_dataset adds for a filename
def __dask_tokenize__(self):
    return normalize_token((type(self).__name__, self._opener, self._args, self._mode, self._kwargs,
                            _get_mtime(self._args[0]) if self._args else None))

# NetCDF4DataStore, H5NetCDFStore (and ScipyDataStore)
def __dask_tokenize__(self):
    return normalize_token((type(self).__name__, self._manager, self._group))

Describe alternatives you've considered

My Claude-based workaround is to define a wrapper store object that defines the __dask_tokenize__ method myself. It's a bit hacky and dependent on xarray not changing the interfaces to the store too much.

Additional context

No response