Repository navigation
Description
On the native wheel, NativeAsyncExecutor.append_samples turns every column into a Python list and sends the whole chunk as one JSON text. plain() runs .tolist() on each numpy column, stamps = [int(t) for t in plain(timestamps)] walks the timestamps a second time, and _op("append_samples", type=, ts=, cols=) encodes everything with _enc (the _prep walk plus json.dumps) for arcadedb_n_async. NativeExtras.async parses the text back into Object[] columns. The primitive=True flag is only a length check on the native build.
The JVM wheel does the opposite for the same call: _append_samples_primitive sends one JNI array per numeric column through TimeSeriesBatcher, and only the string tag column goes element by element. The library already has binary routes for other calls (arcadedb_n_exec_bin takes a float32 side buffer and arcadedb_n_insert_colp takes an address table), arcadedb_n_async has none.
Measurement (untimed, laptop, CI wheels of main 413f54f, native and JVM 26.11.1.dev0, one run per cell, so orientation and not a ratio to quote). The shape is the l4 time-series lane's: 1,000,000 points, 100 hosts, 100,000-point chunks, numpy int64 and float64 columns, a Python list of host strings, primitive=True, one wait_completion() at the end. Both runs read the stored samples back by count, sums and windows and match the source.
| figure (lane shape, one run, untimed) | native wheel | JVM wheel |
|---|---|---|
| append calls, us per point | 0.755 | 0.314 |
| whole load wall, s (includes the harness's own column building) | 0.957 | 0.736 |
Python glue (tolist() of every column and the int() loop), us per point |
0.100 | n/a |
JSON encode (extras._enc), us per point |
0.347 | n/a |
crossing into arcadedb_n_async (Java parse and queue), us per point |
0.318 | n/a |
| glue plus encode, share of the load wall | 46.7% | n/a |
The text is 43.6 bytes per point, against 38.9 bytes for the same columns as raw int64 and float64 plus the host text, so the size is not the cost: formatting, parsing and the two list conversions are. Glue plus encode is 0.447 of the 0.755 us per point of the append calls, and 46.7% of the load wall (median chunk of the run: append 75.6 ms, encode 34.7 ms). The crossing figure also holds the Java parse and any wait for the writers, so a binary route may move it too; that is for the A/B to say.
Proposal: add arcadedb_n_async_bin(handle, op, json, table, n_table) next to arcadedb_n_async, following the address-table convention of arcadedb_n_insert_colp. For append_samples, a one-dimensional numpy int or float column (and an integer or datetime64[ns] timestamp array) crosses by address as int64 or float64 and the JSON carries only a kind marker per column; the host tag column and anything else stays as text. Java builds the same long[] and boxed Object[] columns and calls the same appendSamples, so the stored samples are bit-identical to today (a float32 array widens to float64, bool is 0 and 1, NaN is the canonical NaN the text path gives). A library without the entry, or ARCADEDB_NATIVE_TS_BIN=0, keeps the JSON path. Lists, float arrays as timestamps, 2-D arrays, uint64 past the int64 range and a column whose length differs from the timestamps keep the JSON path. primitive=True keeps its meaning (the same length check). The JVM wheel is not touched.
Not claimed: the table is a probe, not a timed A/B. A wheel-versus-wheel A/B with the usual repetitions comes after the PR's CI wheel exists. The Java side still boxes each value once; the engine's TimeSeriesBatch route is a possible follow-up, not part of this change.
Activity
Native append_samples tests: fix three wrong expectations of the new …