Skip to content

perf: decode cached batches in native code instead of on the JVM #7296

Open

Description

@andygrove

What is the problem the feature request solves?

Part of #6834.

Reading Comet's in-memory cache is JVM-only work before any data reaches native code. ArrowCachedBatchSerializer copies the cached body from heap to off-heap, inflates zstd through zstd-jni, runs Arrow's VectorLoader, wraps the vectors in CometVectors, and only then exports them over FFI. The write side is the same shape: column statistics and compression both run on the JVM.

When the consumer is a native plan, none of that has to be JVM work. Handing the IPC bytes to native and decoding them with arrow-rs would remove the heap-to-off-heap copy, the JVM decompression and the intermediate vector allocation from every cached read, and the same applies to statistics and compression when a batch is cached.

Describe the potential solution

Add a native decode path for cached batches, used when the scan feeds a native operator, and measure it against the current path with the existing cache benchmark before deciding whether to move the write side as well. It fits alongside the prefetch work in #5487. The Spark-consumer case in #5485 still needs the JVM vectors, so the JVM path stays for it.

Additional context

Found in the cache audit (#6834).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions