Skip to content

JVM create_vertices: float[] vectors from to_java_float_array decline the json-bulk matrix path #220

Closed as not planned

Description

@tae898

On the JVM wheel, GraphBatch.create_vertices sends float vectors given as JPype float[] (what arcadedb.to_java_float_array returns there) through the slow property-matrix path instead of the json-bulk plus binary float32 matrix path.

is_vector_candidate in src/arcadedb/type_conversion.py accepts only a list or a numpy array. _create_vertices_json_bulk in src/arcadedb/graph_batch.py then sees a value that is not a vector candidate and not a JSON scalar (json_bulk_scalar_ok), returns None, and the whole chunk goes through _to_java_property_matrix (per-value Java marshalling). The same candidate test is shared by Database.insert_many through split_float_vectors, so a float[] vector there stays out of the matrix too. insert_columns takes whole columns and does not use the candidate test. On the native wheel to_java_float_array already returns a float32 ndarray (vector.py, the _native_enabled() branch), so nothing changes there.

The benchmark harnesses build every vector with to_java_float_array, so their create_vertices calls take the declined path on the JVM wheel. Probe: probe_vertices.py on the laptop, JVM wheel 26.11.1.dev0 built from main 413f54f, 30,000 vertices (20,000 for e2_64), create_vertices in chunks, declared ARRAY_OF_FLOATS column, one run per variant. These are UNTIMED orientation figures (no ab.py, no timed lock, one run), not a speed claim; the numbers below are read from the probe log by a script.

shape dim path, harness path, ndarray load us/vertex, harness load us/vertex, ndarray ratio
l3d_sift 128 json-bulk DECLINED -> _to_java_property_matrix json-bulk + binary float32 matrix 21.72 9.17 2.37x
l3d_deep 96 json-bulk DECLINED -> _to_java_property_matrix json-bulk + binary float32 matrix 11.83 5.87 2.02x
e2_64 64 json-bulk DECLINED -> _to_java_property_matrix json-bulk + binary float32 matrix 13.04 5.88 2.22x

harness builds to_java_float_array(vec) per vector, ndarray passes the float32 row itself. load us/vertex is create_vertices plus the graph batch close, the row build is not in it. The harness shape also pays its own per-vector JArray construction in the row build (2.6 to 4.6 us per vertex more than the ndarray row build in the same log); that part stays with the caller.

Proposed change: accept a JPype float[] or double[] as a vector candidate. numpy reads both through the JPype buffer protocol (np.array of a column of float[] gives a float32 matrix, double[] gives float64, which split_float_vectors already casts to float32 as it does for float64 ndarrays), so the existing split code needs no new conversion call. A column whose first value is a Java array must have only candidates in it; a JArray(JInt) or JArray(JLong) keeps declining, and so do a null, a missing value, a ragged vector, or an empty array, as today. Stored values are bit-identical to the declined path (tested against it). Tests assert the path with spies on split_float_vectors and _to_java_property_matrix.

A timed one-library A/B on the laptop (arm A: the main CI wheel, arm B: the same wheel with only the changed module overlaid, jars identical) follows once the PR is green.

Activity

  1. tae898 commented on Nov 5, 2026

    @tae898
    CollaboratorAuthor

    Closing as not planned: the change in #221 was measured twice and came out inconclusive by its pre-registered rule (results on #221), and speed was its only motive, so the current path stays.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions