Skip to content

Native to_json_list walks untagged rows again, and json_native returns RID objects json.dumps refuses #219

Closed

Description

@tae898

Problem

Two things in the native NativeResultSet.to_json_list (src/arcadedb/_native/database.py) and json_native (src/arcadedb/_native/codec.py).

1. A second walk over rows that cannot change. to_json_list drains the decoded rows and then runs json_native over every value of every row. codec.loads already skips its per-object hook when the reply text holds no escaped tag (_TAG_ESCAPED not in text), and json.loads alone returns only str, int, float, bool, None, list and dict, which json_native returns unchanged. For a result in which no reply carried a tagged value the walk is pure overhead.

The numbers below are from a probe on the laptop, native wheel 26.11.1.dev0, cores 8 to 15, one process, untimed (orientation only, not an A/B; the timed A/B is separate). Medians over repetitions, paper-shaped results from a 10,000-vertex fixture. Every reply was untagged, and to_list, to_json_list and the identity-patched to_json_list returned equal rows.

shape rows x cols tagged replies to_list us to_json_list us walk us walk share of to_json_list
point 1 x 2 0/1 1.03 2.26 1.24 55.4%
top10_3col 10 x 3 0/1 2.13 11.71 9.84 83.0%
hist100 100 x 2 0/1 4.96 68.70 63.96 92.9%
scan4_r100 100 x 4 0/1 5.08 107.71 103.06 95.4%
sparse10 10 x 2 0/1 1.38 8.20 7.00 84.1%

The walk is 51% to 95% of the to_json_list wall on the shapes up to 100 rows (the statement is not in this table; the walk is 0.0% to 5.7% of statement plus drain plus walk on the shapes up to 100 rows, so for the engine-bound reads the saving is small and for a short indexed read it is a few percent). Larger results pay it in proportion to rows (1.03 to 1.06 us per row of 4 columns on the 100 to 10,000 row scans).

2. json_native returns values json.dumps refuses. to_json_list is the method that promises JSON-native rows, and the JVM wheel returns the RID text ("#1:0") for a link, in a column, in a list and inside a map (checked 2026-11-04 on the 26.11.1.dev0 JVM and native wheels with a LINK property, a list of links, SELECT @rid, and a map value). The native wheel returns the RID object. The probe's unit cases on json_native (FINDING lines):

case json.dumps
RID value TypeError: Object of type RID is not JSON serializable
RID in list TypeError: Object of type RID is not JSON serializable
map with RID key TypeError: keys must be str, int, float, bool or None, not RID

A map whose keys are not strings also keeps its keys on native (a MAP property with integer keys comes back as {1: 'a'}), where the JVM wheel returns {'1': 'a'} (a JSON object has string keys).

Proposed change

  • codec.loads_tagged returns the reply and whether a tagged value was restored; the result set ORs that flag over every reply it decodes (first batch, arcadedb_n_next, arcadedb_n_next_bin). to_json_list skips json_native when the flag is false. A binary reply whose only tags are ["B", offset, count] decodes to plain float lists and does not need the walk.
  • json_native turns an RID into its text and map keys into strings, as the JVM wheel does.
  • No change to to_list; to_json_list output stays identical for the tagged cases that were JSON-safe (DATE and DATETIME as epoch milliseconds, DECIMAL as float, sets as lists).

Source: probe probe_json_list.py and its out-20261103/native-json_list.log (HumemAI notes, not public); the figures above were extracted from that log by script.

Activity

  1. added this to the 26.11.1 milestone on Nov 5, 2026
  2. added
    nativeThe arcadedb package (native library build)
    performanceSpeed or memory of the bindings, with a measurement
    on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    nativeThe arcadedb package (native library build)performanceSpeed or memory of the bindings, with a measurement

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions