Skip to content

JVM wheel: record and row readers cross into the JVM many times (to_dict, property names, result-set iteration) #227

Closed

Description

@tae898

On the JVM wheel (arcadedb-jvm), reading a record or a row into a Python dict, asking for its property names, and iterating a result set all cross into the JVM many times per call, where the native build crosses zero times. A JPype call costs a few microseconds of dispatch, so on a small record the number of calls, not the data, is the cost.

Crossing counts from an untimed census of the JVM wheel built from main 413f54ffe1 (warm calls, a lower bound: attribute reads, jarray[i] and for x in java_iterable are not counted; the files are d2/census-jvm.tsv and d2/census-native.tsv of the discovery pass of 2026-11-03, read by a script). These are counts, not timings; no timed run exists yet.

call scenario JVM crossings per unit native crossings per unit
Document.to_dict 3 properties 18.00 0.00
Result.to_dict 3 properties 14.00 0.00
Document.get_property_names 3 properties 12.00 0.00
Result.get_property_names 3 properties 12.00 0.00
Result.property_names property 12.00 0.00
ResultSet.__iter__ for r in rs, 10 rows 2.20 0.00

What each one does today:

  • Document.to_dict() asks for getPropertyNames(), walks the Java set element by element, then makes one get(name) call per property, and converts each value with its own JPype reads.
  • Result.to_dict() already takes the whole row in one call (RowAccess.namesAndValues), but then reads every name and every value of the returned arrays one JPype object at a time.
  • get_property_names() and property_names (for both classes) walk the Java set element by element, until the names are cached.
  • ResultSet.__next__ asks hasNext() and then next(): two crossings per row, and one more at the end.

The fix is one new bridge class in src/java (a sibling of RowAccess and TypedRows, so it ships in the same jar and the wheel build compiles it with the others): a row or a record is written as the typed JSON of TypedRows (the transport ResultSet.to_list() already uses, with the same tags and the same side array for the engine's own objects), property names come back as one JSON text, and nextOrNull does hasNext() and next() in one call. The values keep the types convert_java_to_python gives them today, the key order is the engine's, a result set is still read one row per call (no batch is taken ahead), and the native build is not touched.

Tests that count the calls into the JVM (a counting proxy on the bridge class and the Java objects) and compare every value type, the key order, the errors and the result-set state with the plain path are part of the PR. A timed A/B (main CI wheel against the PR's CI wheel, with an A/A control and a verdict rule written before the first run) is prepared, and the PR should not leave draft on the counts alone.

Overlap: PR #214 (issue #213) also edits RowAccess.java, results.py (ResultSet.one, count) and graph.py (Document.get_raw). This change adds its Java in a new file and edits ResultSet.__next__, Result.to_dict, Result._property_names_tuple, Document.to_dict and Document._property_names_tuple, so the two should merge without a conflict in either order.

🤖 Generated with Claude Code

Activity

  1. tae898 commented on Nov 4, 2026

    @tae898
    CollaboratorAuthor

    Timed result for this issue: PR #233 is a SHIP by the rule written before the run (record-to-dict and row-to-dict faster on every gating workload of the JVM wheel, the control flat, a clean A/A). Figures with their 95% intervals: #233 (comment)

    Edited 2026-11-05: the speed-up is now given only as the intervals in the linked comment.

  2. added this to the 26.11.1 milestone on Nov 5, 2026
  3. added
    jvmThe arcadedb-jvm package (JPype build)
    performanceSpeed or memory of the bindings, with a measurement
    on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    jvmThe arcadedb-jvm package (JPype build)performanceSpeed or memory of the bindings, with a measurement

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions