Repository navigation
Description
Describe the bug, including details regarding any error messages, version, and platform.
Disclaimer: Claude Opus 5 found this bug.
Describe the bug
BaseVariableWidthViewVector.splitAndTransferViewBufferAndDataBuffer allocates the target's view
buffer without zeroing it:
target.viewBuffer = target.allocator.buffer(length * ELEMENT_SIZE); // line 888 at v19.0.0For an inlined value (12 bytes or fewer) it then writes only the 4-byte length and the value's
bytes, so the rest of the 16-byte view keeps whatever the allocation held. The Arrow columnar
format requires those padding bytes to be zero, and setBytes already does this (it zeroes the
view before writing an inlined value), but this path does not.
So any splitAndTransfer of a ViewVarCharVector / ViewVarBinaryVector can produce invalid
views. That includes TransferPair.splitAndTransfer and VectorSchemaRoot.slice(). Consumers
that validate are then right to reject the data. For example, after export through the C Data
Interface, arrow-rs's ArrayData::validate_full fails with:
View at index 0 contained non-zero padding for string of length 0
To reproduce
import java.nio.charset.StandardCharsets;
import org.apache.arrow.memory.ArrowBuf;
import org.apache.arrow.memory.BufferAllocator;
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.ViewVarCharVector;
import org.apache.arrow.vector.util.TransferPair;
public class PaddingRepro {
public static void main(String[] args) {
try (BufferAllocator allocator = new RootAllocator();
ViewVarCharVector v = new ViewVarCharVector("s", allocator)) {
v.allocateNew();
v.setSafe(0, "a longer value".getBytes(StandardCharsets.UTF_8));
v.setSafe(1, "".getBytes(StandardCharsets.UTF_8));
v.setValueCount(2);
TransferPair tp = v.getTransferPair(allocator);
tp.splitAndTransfer(1, 1);
ArrowBuf views = ((ViewVarCharVector) tp.getTo()).getDataBuffer();
for (int i = 4; i < 16; i++) {
System.out.print(views.getByte(i) + " "); // expected all 0
}
System.out.println();
tp.getTo().close();
}
}
}With arrow-vector 19.0.0 on HotSpot 21 this prints (the exact bytes vary):
-112 127 0 0 -112 0 0 8 -112 127 0 0
Expected behavior
All 12 bytes after the length of an inlined view are zero after splitAndTransfer, for example
by zeroing the target's view buffer after allocating it (setZero), as setBytes does for a
single view. Present at v19.0.0 and unchanged on main.
Component(s)
Java (arrow-vector 19.0.0)
Activity
hutiefang76 commentedon Oct 24, 2026
I can take this. I'll keep the fix in the view-vector transfer path and add a regression using memory filled with non-zero bytes, so it does not depend on allocator reuse. The test will cover UTF-8 and binary views, including empty and boundary-length values, with controls for long values and nulls.
AI-generated coordination comment; source investigation and implementation assisted by Codex and DeepSeek.