Performance: numpy indexes small amounts of data 1000 faster than xarray

Machine learning applications often require iterating over every index along some of the dimensions of a dataset. For instance, iterating over all the (lat, lon) pairs in a 4D dataset with dimensions (time, level, lat, lon). Unfortunately, this is very slow with xarray objects compared to numpy (or h5py) arrays. When the Pangeo machine learning working group met today, we found that several of us have struggled with this.

I made some simplified benchmarks, which show that xarray is about 1000 times slower than numpy when repeatedly grabbing a small amount of data from an array. This is a problem with both isel or [] indexing. After doing some profiling, the main culprits seem to be xarray routines like _validate_indexers and _broadcast_indexes.

While python will always be slower than C when iterating over an array in this fashion, I would hope that xarray could be nearly as fast as numpy. I am not sure what the best way to improve this is though.

@rabernat

cc @rabernat

While python will always be slower than C when iterating over an array in this fashion, I would hope that xarray could be nearly as fast as numpy. I am not sure what the best way to improve this is though.

I'm sure it's possible to optimize this significantly, but short of rewriting this logic in a lower level language it's pretty much impossible to match the speed of NumPy.

This benchmark might give some useful context:

def dummy_isel(*args, **kwargs):
  pass

def index_dummy(named_indices, arr):
    for named_index in named_indices:
        dummy_isel(arr, **named_index)

%%timeit -n 10
index_dummy(named_indices, arr)

On my machine, this is already twice as slow as your NumPy benchmark (497 µs vs 251 µs) , and all it's doing is parsing *args and **kwargs! Every Python function/method call involving keyword arguments adds about 0.5 ns of overhead, because the highly optimized dict is (relatively) slow compared to positional arguments. In my experience it is almost impossible to get the overhead of a Python function call below a few microseconds.

Right now we're at about 130 µs per indexing operation. In the best case, we might make this 10x faster but even that would be quite challenging, e.g., consider that even creating a DataArray takes about 20 µs.

@shoyer

Thanks so much @shoyer. I didn't realize there was that much overhead for a single function call. OTOH, 2x slower than numpy would be way better than 1000x.

After looking at the profiling info more, I tend to agree with your 10x maximum speed-up. A couple of particularly slow functions (e.g. Dataset._validate_indexers) account for about 75% of run time. However, the remaining 25% is split across several other pure python routines.

To be clear, pull requests improving performance (without significantly loss of readability) would be very welcome. Be sure to include a new benchmark in our benchmark suite.

@nbren12

Thanks for the benchmarks @nbren12, and for the clear explanation @shoyer

While we could do some performance work on that loop, I think we're likely to see a material change by enabling the external library to access directly from the array, without a looped python call. That's consistent with the ideas @jhamman had a few days ago.

@max-sixty

@max-sixty I tend to agree this use case could be outside of the scope of xarray. It sounds like significant progress might require re-implementing core xarray objects in C/Cython. Without more than 10x improvement, I would probably just continue using numpy arrays.

You can always use xarray to process the data, and then extract the underlying array (da.values) for passing into something expecting an numpy array / for running fast(ish) loops (we do this frequently).

Sure, I've been using that as a workaround as well. Unfortunately, that approach throws away all the nice info (e.g. metadata, coordinate) that xarray objects have and requires duplicating much of xarray's indexing logic.

To put the relative speed of numpy access into perspective, I found this insightful: https://jakevdp.github.io/blog/2012/08/08/memoryview-benchmarks/ (it's now a few years out of date, but I think the fundamentals still stand)

Pasted from there:

Summary
Here are the timing results we've seen above:

Python + numpy: 6510 ms
Cython + numpy: 668 ms
Cython + memviews (slicing): 22 ms
Cython + raw pointers: 2.47 ms
Cython + memviews (no slicing): 2.45 ms

So if we're running an inner loop on an array, accessing it using numpy in python is an order of magnitude slower than accessing it using numpy in C (and that's an order of magnitude slower than using a slice, and that's an order of magnitude slower than using raw pointers)

So - let's definitely speed xarray up (your benchmarks are excellent, thank you again, and I think you're right there are opportunities for significant increases). But where speed is paramount above all else, we shouldn't use any access in python, let alone the niceties of xarray access.

Cython + memoryviews isn't quite the right comparison here. I'm sure ordering here is correct, but relative magnitude of the performance difference should be smaller.

Xarray's core is bottlenecked on:

Overhead of abstraction with normal Python operations (e.g., function calls) in non-numeric code (all the heavy numerics is offloaded to NumPy or pandas).
The dynamic nature of our APIs, which means we need to do lots of type checking. Notice how high up builtins.isinstance appears in that performance profile!

C++ offers very low-cost abstraction but dynamism is still slow. Even then, compilers are much better at speeding up tight numeric loops than complex domain logic.

As a point of reference, it would be interesting to see these performance numbers running pypy, which I think should be able to handle everything in xarray. You'll note that pypy is something like 7x faster than CPython in their benchmark suite, which I suspect is closer to what we'd see if we wrote xarray's core in a language like C++, e.g., as Python interface to xframe.

Cython + memoryviews isn't quite the right comparison here.

Right, tbc, I'm only referring to the top two lines of the pasted benchmark; i.e. once we enter python (even if only to access a numpy array) we're already losing a lot of the speed relative to the loop staying in C / Cython. So even if xarray were a python front-end to a C++ library, it still wouldn't be competitive if performance were paramount.
...unless pypy sped that up; I'd be v interested to see.

It might be interesting to see, if pythran is an alternative to Cython. It seems like it handles high level numpy quite well, and would retain the readability of Python. Of course, it has its own issues...

But it seems like other libraries like e.g. scikit-image made some good experience with it.

Sadly I can't be of much help, as I lack experience (and most importantly time).

Pythran supports Python 2.7 and also has a decent Python 3 support.
[...]
Pythran now supports Python3 and can be installed as a regular Python3 program. Note however that Python3 support is still in early stage and compilation failure may happen. Report them!

This is not a great start :(

It's the first time I hear about Pythran. At first sight it looks somewhat like a hybrid between Cython (for the ahead-of-time transpiling to C++) and numba (for having python-compatible syntax).

That said, I didn't see anything that hints at potential speedups on the python boilerplate code.

I already had experience with compiling pure-python code (tight __iter__ methods) with Cython, and got around 30% performance boost which - while nothing to scoff at - is not life-changing either.

This said, I'd have to spend more time on it to get a more informed opinion.

At first sight it looks somewhat like a hybrid between Cython (for the ahead-of-time transpiling to C++) and numba (for having python-compatible syntax).

Not really. Pythran always releases the GIL and does a bunch of optimizations between transpilation and compilations.

A good approach would be try out different compilers and see what performance is obtained, without losing readability (#2799 (comment)). See scikit-image/scikit-image/issues/4199 where the package transonic was being experimentally tested to replace Cython-only code with python code + type hints. As a bonus, you get to switch between Cython, Pythran and Numba,

@jhamman

@jhamman Weren't you talking about an xarray lite (TM) package?

I agree, I think a "xarray lite" package with only named dimensions could indeed be a valuable contribution.

I'd love to optimize xarray further, but I suspect you would probably have to write the core in a language like C++ to achieve similar performance to NumPy.

@nbren12

I hope the following can help users that struggle with the speed of xarray:

I've found that when doing numerical computation, I often use the xarray to grab all the metadata relevant to my computation. Scale, chromaticity, experimental information.

Eventually, i create a function that acts as a barrier:

Xarray input (high level experimental data)
Computation parameters output (low level implementation detail relevant information).

The low level implementation can operate on the fast numpy arrays. I've found this to be the struggle with creating high level APIs that do things like sanitize inputs (xarray routines like _validate_indexers and _broadcast_indexes) and low level APIs that are simply interested in moving and computing data.

For the example that @nbren12 brought up originally, it might be better to create xarray routines (if they don't exist already) that can create fast iterators for the underlying numpy arrays given a set of dimensions that the user cares about.

I agree, I think a "xarray lite" package with only named dimensions could indeed be a valuable contribution.

FWIW, I think the xarray-lite concept would be a great chunk of work to write a small-ish proposal around. I think we could target the next round of CZI EOSS with such a concept.

@hmaarrfk

Thanks all for your prompt responses!

@hmaarrfk , I share your recommendation and it's a great thing to be able to fallback to numpy arrays when the algorithmic part is well decoupled from the data preparation process. It's what I also do when I can.
However, in workflows working on streaming data the two things (data preparation and computation) may be intricated or frequently alternated. My example of "buffer data array" structure is something quite natural to consider in such a context and having an efficient implementation of labelled ndarray could really serve the task.

@shoyer I think a first "lite" implementation fully implemented in python could be already a great thing.
It would not achieve numpy performance, but the additional cost du to management of coordinates alignement should not be too expensive.

An additional suggestion: if the target is computational workflows, trying to have some compatibility with packages such as eagerpy would enabling working with other tensor frameworks commonly used in machine learning.
This kind of feature could be adressed yet in another package, but having it in mind may influence the early choices in term of implementation (ex: pure python vs C++).

@jhamman, @shoyer I would be pleased to share my work on buffer data array if you think it could serve as kind of use-case. In this context, I experimented a bit with a « crafted » lite version of xarray and I could achieve a x10 factor in performance improvement.

In case it could be usefull, and be reused for benchmarks, I released on my github two notebooks with an implementation of a faster (but somehow very simplified and not very optimized in term of code architecture) version of DataArray and Dataset containers. The second notebook contains some line profilings for buffer experiments with various containers. This permits to point on operations which are slow in Datarray implementation for this use case.

@jhamman

FWIW, I think the xarray-lite concept would be a great chunk of work to write a small-ish proposal around. I think we could target the next round of CZI EOSS with such a concept.

@jhamman I'll be happy to participate in the discussion.

Hello,

I don't want to disrupt the issue too much (so let me know if you'd rather we continue the discussion outside).

Somewhat related to the discussions in this issue, I recently released an open-source library: WAX-ML, https://github.com/eserie/wax-ml, where I implement an accessor to unroll JAX transformations on DataSet and Dataarray xarray containers along a time dimension.

I hope this can help!

I'm really not understanding why indexing is so slow. My dataarray has 2 dims, one axis 1.5 million long ('node') and the other 1500 ('time'). Trying to pull a single timeseries by indexing 1 node takes 16 seconds. the Variable workaround or playing around with chunking doesn't change anything. The only thing loading into memory should be array of 1500 values.

Not sure what's going on under the hood but there may be a way to specify that you're only looking to optimize indexing along 1 dim. Once it gets indexed it becomes a very tiny data set. I would think chunks={'node':1} would do exactly this but I guess not.

As I've been recently going down this performance rabbit hole, I think the discussion around #7045 is relevant and provides some additional historical context as to "why" this performance penalty might be happening.

So a workaround I was able to use was to load the whole thing into a np array (18GB!) in 1 minute da.values, index 15 nodes in 0.4 seconds (was taking ~5min in xarray), then load it back into a dataarray. Not accommodating for our friends of differently-abled memory cards, but it worked in my case

You all know this, but I didn't realize how very slow read gunzip / write gzip can be.
From a simple testbench, here with parameter gb=2:

# from: xrtestbench.py
# 12 Sep 2023 09:44z  ~bz np/time/xr

versions np 1.25.2  xr 2023.8.0  netCDF4 1.6.4  python 3.10.0  

Randomcube (8760, 247, 247)  2039 mb 

time:   7.9 sec  0.3 cpus  np.savez "/tmp/cube.npz", **cube._asdict()

time:   3.1 sec  0.8 cpus  np.load( "/tmp/cube.npz", mmap_mode=None ) .f.values
time:   2.4 sec  1.1 cpus  np.load( "/tmp/cube.npz", mmap_mode=None ) .f.values
time:   2.6 sec  1.1 cpus  np.load( "/tmp/cube.npz", mmap_mode=None ) .f.values

-- complevel 0
time:   7.9 sec  0.2 cpus  xds.to_netcdf( /tmp/0.nc )  2039 mb  complevel 0

time:   4.5 sec  1.0 cpus  xr.load_dataarray( /tmp/0.nc ) .to_numpy()
time:   1.7 sec  1.0 cpus  xr.load_dataarray( /tmp/0.nc ) .to_numpy()
time:   1.8 sec  1.0 cpus  xr.load_dataarray( /tmp/0.nc ) .to_numpy()

-- complevel 1
time:  64.0 sec  1.0 cpus  xds.to_netcdf( /tmp/1.nc )  1687 mb  complevel 1

time:  14.7 sec  0.9 cpus  xr.load_dataarray( /tmp/1.nc ) .to_numpy()
time:  13.4 sec  1.0 cpus  xr.load_dataarray( /tmp/1.nc ) .to_numpy()
time:  13.1 sec  1.0 cpus  xr.load_dataarray( /tmp/1.nc ) .to_numpy()

Here xarray and numpy times, both with no compression, are not so far apart --
on my old 2.7 GHz imac with 16 GB, SSD caching ? dunno.
(On time ratios, can't resist: https://imgs.xkcd.com/comics/types_of_approximation.png )
Anyone care to rerun this on a modern machine, ideally on a free cloud server for reproducibility ?

Not surprisingly, xarray with complevel 1 is terribly slow, gunzip / gzip.
(Beware that hdf5 parameters complevel etc. are by default sticky:
ar = xr.load_array compressed.nc => ar.to_netcdf( no encoding= ) is also compressed.
Hdf5 has lots of possibilities, anyone know of concise doc ?)

Two obvious suggestions:

don't compress gigabyte files if you don't have to
if you get big compressed files, uncompress them right after downloading with
nccopy -d 0 comp.ncin nocomp.ncout or the like. This will save time downstream.

perf(napari): pass raw array to napari.imshow instead of DataArray

Improve performance by changing the accessors (.values) pydata/xarray…

Navigation Menu

Performance: numpy indexes small amounts of data 1000 faster than xarray #2799

15 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Performance: numpy indexes small amounts of data 1000 faster than xarray #2799

Description

Activity

nbren12 commented on Mar 4, 2019

shoyer commented on Mar 4, 2019

nbren12 commented on Mar 4, 2019

shoyer commented on Mar 4, 2019

max-sixty commented on Mar 4, 2019

nbren12 commented on Mar 4, 2019

max-sixty commented on Mar 4, 2019

nbren12 commented on Mar 4, 2019

max-sixty commented on Mar 5, 2019

shoyer commented on Mar 5, 2019

max-sixty commented on Mar 5, 2019

DerWeh commented on Sep 9, 2019

crusaderky commented on Sep 9, 2019

ashwinvis commented on Oct 4, 2019

15 remaining items

nbren12 commented on Feb 26, 2021

shoyer commented on Feb 26, 2021

hmaarrfk commented on Feb 26, 2021

jhamman commented on Feb 26, 2021

eserie commented on Feb 26, 2021

eserie commented on Mar 4, 2021

eserie commented on Mar 11, 2021

eserie commented on Jun 17, 2021

openSourcerer9000 commented on Nov 7, 2022

hmaarrfk commented on Nov 7, 2022

openSourcerer9000 commented on Nov 7, 2022

denis-bz commented on Sep 12, 2023

openSourcerer9000 commented on Mar 18, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions