You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
vlen and filter codecs encode F-contiguous arrays in memory order, not logical order #850
The vlen codecs flatten their input with reshape(-1, order='A') before encoding (vlen.pyx lines 90, 220, 364). For an F-contiguous object array that flattens column-major, so the encoded stream holds the elements transposed relative to what a C-order consumer expects on decode:
Delta, FixedScaleOffset and PackBits behave the same way (same order='A' pattern in delta.py, fixedscaleoffset.py, packbits.py and categorize.py), verified with the same kind of round-trip. A non-contiguous view doesn't hit this, since reshape has to copy there and does so in C order; F-contiguous is the one layout where the zero-copy flatten changes element order. zarr v2 was internally consistent because decode wrote through the out= parameter with the same order='A' logic, but any consumer that reshapes the decoded 1-D result in C order gets scrambled data.
Either the C-contiguity requirement gets documented and callers normalize (what the zarr PR does), or encode normalizes internally with np.ascontiguousarray. The second changes what gets written for F-contiguous input, so anything relying on the old out=-symmetric round-trip would read old data back differently; that trade-off is why I'm raising this rather than sending a patch directly.
Happy to send the PR for whichever direction you prefer.
Activity
Fix F-order encode/decode so it follows logical array order
Fix F-order encode/decode so it follows logical array order