You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
JVM GraphBatch: a list, dict or date-time property sends create_vertices and new_edges to per-element Python #243
On the JVM wheel, one list, tuple, dict, date-time, date, Decimal or bytes property in a row moves the whole GraphBatch.create_vertices call, and the whole GraphBatch.new_edges call with properties, off the JSON bulk path onto per-element Python.
_create_vertices_json_bulk in src/arcadedb/graph_batch.py returns None as soon as json_bulk_scalar_ok refuses one value, and the chunk goes through _to_java_property_matrix (one JPype conversion per element). new_edges does the same and then buffers every edge with new_edge (one crossing and one RID build per edge). #82 added tagged values to the JSON text for insert_many (json_bulk_dumps_tagged, decoded by DocumentBatcher.untag), so the text can already carry a date-time, date, Decimal and bytes that sit directly in a row, and any nested list or map of JSON values. The graph batches kept the untagged check.
Evidence so far is UNTIMED orientation only: a list property cost 91.4 us per row against 9.3 us for the same rows without it (one run, no timed lock). Scalar-only properties are engine bound (profile of ingest_graph on the JVM wheel: engine 80.7% of the calling thread, Java glue 1.7%), so only these shapes are affected. #496 (#33) covered declared ARRAY_OF_FLOATS vectors only.
Proposed change, in a draft PR: carry those shapes through json_bulk_dumps_tagged, decode them in VertexBatcher and EdgeBatcher with DocumentBatcher.untag (and toMap(), so a nested list or map is stored as a plain List or Map as insert_many does), and keep everything the text cannot carry on the old path: a NaN, an integer beyond 64 bits or a date-time inside a list or map, a subnormal, a non-str dict key, a set, a numpy or pandas value. The stored values must equal what the per-element path stores, types included; the PR's tests compare both.
A timed A/B of the PR (wheel against wheel, since the bridge jar changes) follows once CI has built the wheels.
Activity
JVM GraphBatch: numeric lists and all-scalar inputs keep main's path (#…
TA-6 docs: which property shapes a JVM graph batch loads in bulk, the…