Inconsistency in whether index is created with new dimension coordinate? #4417

Activity

1. johnomotani

   added a commit that references this issue on Sep 10, 2020

   Change creation of time coordinate to ensure creation of 'index'

   ...

   c346c46
2. max-sixty commented on Sep 11, 2020

   

   max-sixty

   on Sep 11, 2020

   Collaborator

   More actions

   Thanks for the clear example @johnomotani !

   IIRC, the second creates an index because ds['x'] = ('x', np.linspace(3,4)) is setting a 1D variable where the name of the variable matches the name of its dimension.

   I can see why this is a bit confusing, though it's not an unreasonable default imo. There have been some discussions on this on the issue tracker.

   Does ds.set_index(x='b') align the examples? Or do you think there's still some confusion from the API?
3. johnomotani commented on Sep 11, 2020

   

   johnomotani

   on Sep 11, 2020

   ContributorAuthor

   More actions

   I haven't managed to get set_index to line up the results. This:

   import numpy as np
   import xarray as xr
   
   ds = xr.Dataset()
   ds['a'] = ('x', np.linspace(0,1))
   ds['b'] = ('x', np.linspace(3,4))
   ds = ds.rename(b='x')
   ds = ds.set_coords('x')
   ds = ds.set_index(x='x')
   
   print(ds)
   print('indexes', ds.indexes)
   

   Produces a Dataset that now has an index x, but no x coordinate. The 'assignment of 1D variable' version produces both a coordinate and an index.

   One suggested solution in #2461 was to use swap_dims(), and the following does produce the desired result

   import numpy as np
   import xarray as xr
   
   ds = xr.Dataset()
   ds['a'] = ('x', np.linspace(0,1))
   ds['b'] = ('x', np.linspace(3,4))
   ds = ds.swap_dims({'x': 'b'})
   ds = ds.rename(b='x')
   
   print(ds)
   print('indexes', ds.indexes)
   

   But (1) having to use swap_dims() to create a coordinate seems bizarre, (2) I think it's a bit ugly to get rid of the x dimension with swap_dims(), and then have to rename the new dimension back to x, when what I wanted was to add a coordinate to x.

   I found the behaviour confusing because I wasn't aware of the index variables at all... I create a coordinate with set_coords(); then do a bunch of manipulations involving slicing pieces out of the Dataset and re-combining them with concat (at least I think this is the relevant part of my code...); finally test the result by taking slices again (with isel()) of the result and comparing them to slices of the original Dataset with xarray.testing.assert_identical(), which failed because of the missing indexes. I guess somewhere in the manipulations I did, some operation created the coordinate in a new Dataset object by assignment, at which point it generated an index for the coordinate too.

   I may well be missing other context that makes this an undesirable thing to do, but for my use-case at least, I think it would make more sense if set_coords() created an index if the coordinate is a dimension coordinate (or whatever the actual criterion is for assignment of a 1d variable to create an index).
4. dcherian commented on Sep 11, 2020

   

   dcherian

   on Sep 11, 2020

   Contributor

   More actions

   I think I was trying to fix this in #4108

   👍React with 👍1Reacted by John Omotani
5. johnomotani commented on Sep 11, 2020

   

   johnomotani

   on Sep 11, 2020

   ContributorAuthor

   More actions

   I think I was trying to fix this in #4108

   👍 On a quick glance it looks to me like #4108 would fix my issue here.
6. benbovy commented on Sep 7, 2022

   

   benbovy

   on Sep 7, 2022

   Member

   More actions

   From discussions in #4825 and elsewhere (e.g., #6607 (comment)), could we say that we're going towards a consensus that the index vs. coordinate API should be decoupled wherever possible?

   If that's the case, then set_coords() shouldn't create any index, but it could be chained with set_index() if necessary (or swap_dims() can be used instead).

   Apparently set_coords() never created an index? I tested @johnomotani's example back to v0.14. Before v2022.6.0, there was a source of confusion where the Dataset and DataArray reprs still showed an * for the dimension coordinate while it had no index. This is fixed in v2022.6.0 where the reprs are now consistent: no * and no index. Does this solve the issue here?

   @johnomotani from your example it looks like the two alternatives below would produce the desired result with v2022.6.0:

   • ds.set_index(x='b')
   • ds.rename(b='x').set_coords('x').set_index(x='x')

   I would encourage using the latter. It is more verbose but clearer (each method is explicit).

   Note also that Xarray now also supports non-dimension coordinates with an index. Once #6971 is merged, renaming "b" to "x" won't be needed. Instead you'll be able to do something like:

   ds = xr.Dataset()
   ds['a'] = ('x', np.linspace(0,1))
   ds['b'] = ('x', np.linspace(3,4))
   ds = ds.set_coords('b').set_xindex('b')
   ds.sel(b=...)
   
7. johnomotani commented on Sep 13, 2022

   

   johnomotani

   on Sep 13, 2022

   ContributorAuthor

   More actions

   Thanks @benbovy - with explicit methods now to produce the result with or without index, I think we can close this now 😃
8. johnomotani

   closed this as completedon Sep 13, 2022