.sel does not keep selected coordinate value in case with MultiIndex

.sel method usually keeps selected coordinate value as a scalar coordinate

In[4] ds1 = xr.Dataset({'foo': (('x',), [1, 2, 3])}, {'x': [1, 2, 3], 'y': 'a'})

Out[4]: 
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    y        <U1 'a'
    x        int64 1
Data variables:
    foo      int64 1

But in MultiIndex case, does not.

In[5] ds2 = xr.Dataset({'foo': (('x',), [4, 5, 6])}, {'x': [1, 2, 3], 'y': 'b'})
        ds = xr.concat([ds1, ds2], dim='y').stack(yx=['y', 'x'])
        ds.sel(x=2)
Out[5] 
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
Data variables:
    foo      (y) int64 2 5

x is gone.
Is it a desired behavior?

@benbovy

No, I don't think this was intentional/desired, though it might be slightly tricky to fix this with the current code structure.

@benbovy any thoughts?

For reference:

In [4]: ds
Out[4]:
<xarray.Dataset>
Dimensions:  (yx: 6)
Coordinates:
  * yx       (yx) MultiIndex
  - y        (yx) object 'a' 'a' 'a' 'b' 'b' 'b'
  - x        (yx) int64 1 2 3 1 2 3
Data variables:
    foo      (yx) int64 1 2 3 4 5 6

In [5]: ds.sel(x=2)
Out[5]:
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
Data variables:
    foo      (y) int64 2 5

@shoyer

@shoyer Thanks for the information.
(Sorry for my inaccurate ipython output)

Another related issue is

In[3]: ds.isel(yx=1)
Out[3]: 
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    yx       object ('a', 1)
Data variables:
    foo      int64 1

where MultiIndex structure is totally gone.
(ds.isel(yx=[0, 1]) works perfectly.)

I am now looking into the code. Do you guys have any idea to fix them?
I am thinking to modify Index.remap_label_indexers
so that it also returns selected_dims containing names of dims that should be converted to scalar coordinates.

The current behavior is actually dictated by pandas and I think it was initially intentional (before we could access MultiIndex levels as virtual coordinates).

In the first example MultiIndex.get_loc_level() is called, which doesn't return a MutliIndex:

In [11]: mdix = ds.yx.to_index()

In [12]: midx.get_loc_level(2, level='x')
Out[12]: 
(array([False,  True, False, False,  True, False], dtype=bool),
 Index(['a', 'b'], dtype='object', name='y'))

If we want to keep the MultiIndex in this case, then I think we can just ignore new_indexes returned by indexing.remap_label_indexers and remove Dataset._replace_indexes.

But we still have to choose whether automatically renaming the dimension makes sense or not, i.e., choose between

In [5]: ds.sel(x=2)
Out[5]:
<xarray.Dataset>
Dimensions:  (yx: 2)
Coordinates:
  * yx       MultiIndex 
  - y        (yx) object 'a' 'b'
  - x        (yx) int64 2 2
Data variables:
    foo      (yx) int64 2 5

and

In [5]: ds.sel(x=2)
Out[5]:
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
    x        (y) int64 2 2
Data variables:
    foo      (y) int64 2 5

The second issue is also due to how MultiIndex works in pandas:

In [10]: midx[0]
Out[10]: ('a', 1)

In [11]: midx[[0, 1]]
Out[11]: 
MultiIndex(levels=[['a', 'b'], [1, 2, 3]],
           labels=[[0, 0], [0, 1]],
           names=['y', 'x'])

I'm just wondering if it really makes sense to keep a MultiIndex for a coordinate that is not a dimension anymore like in this case.

@benbovy

@benbovy Thanks for the comment.

Actually, I expected the following behavior

In [1]: ds.sel(x=2)
Out[1]: 
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
    x        int64 2
Data variables:
    foo      (y) int64 2 5

where the selected x is automatically converted to a scalar coordinate.

And also for the second issue, I expected something like

In [2]: ds.isel(yx=0)
Out[2]: 
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    y        <U1 'a'
    x        int64 1
Data variables:
    foo      int64 1

both x and y become scalar coordinates.

Back to the first issue, I personally prefer to the automatic renaming
because it enables the following chain methods,

ds.sel(x=2).isel(y=1)

that might be intuitive especially for the hierarchic dimension structure.

(mmh apparently I forgot to post a 2nd comment I wrote)

I'm just wondering if it really makes sense to keep a MultiIndex for a coordinate that is not a dimension anymore like in this case.

And this is just a nonsense :) Sorry it is late!

Yep I agree it would be nice to have scalar coordinates in both cases.

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

As a related issue, we could use an option for .sel to keep the selected variable as a singleton coordinate (an array of shape (1,)) instead of making it a scalar.

.sel() with multi-index: return scalar coords

@fujiisoup

@fujiisoup the behavior that you expect has been implemented in #5692 (I still need to fix/update the tests):

>>> ds.sel(x=2)
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) <U1 'a' 'b'
    x        int64 2
Data variables:
    foo      (y) int64 2 5

>>> ds.isel(yx=0)
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    yx       object ('a', 1)
    y        <U1 'a'
    x        int64 1
Data variables:
    foo      int64 1

Note that for the latter example, the yx coordinate may eventually become irrelevant for multi-indexes (we'll need only the level coordinates).

@gabriel-abrahao feel free to open a new issue for your suggestion.

Explicit indexes (#5692)

Navigation Menu

Repository navigation

.sel does not keep selected coordinate value in case with MultiIndex #1408

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

.sel does not keep selected coordinate value in case with MultiIndex #1408

Description

Activity

shoyer commented on May 13, 2017

fujiisoup commented on May 14, 2017

benbovy commented on May 15, 2017

fujiisoup commented on May 15, 2017

benbovy commented on May 15, 2017

stale commented on Sep 19, 2020

gabriel-abrahao commented on Nov 18, 2020

benbovy commented on Sep 15, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions