You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Native insert_many: date-times, Decimals, bytes, numpy scalars and pandas NA send the whole chunk through INSERT ... CONTENT per row #217
On the native build, insert_many sends a whole 10,000-row chunk through INSERT INTO t CONTENT :row for each row (arcadedb_n_exec_many) as soon as one value is a datetime, date, Decimal, bytes, a numpy scalar, or a pandas NA/NaT. The chunk takes that path because _native/database.py::_insert_chunk calls the untagged json_bulk_dumps, which raises on those types. columnar.pack refuses date-times and Decimals too, so the columnar entry does not help.
The JVM build stopped doing this in #82 (json_bulk_dumps_tagged, used by core.py only). Json.java in the native image already decodes the tagged values (T, D, n, b), so the native build needs a Python change only, no native-image rebuild. This was first done in humemai/arcadedb-embedded-python#532, which was closed unmerged when the work moved here.
Untimed, one run each (so no claim): 1,000,000 rows of six properties with a Python datetime per row took 15.2 s through arcadedb_n_exec_many (100 crossings) and 11.2 s through arcadedb_n_insert_rows (100 crossings). A separate untimed probe on the laptop (native, none/one/all rows carrying a datetime) gave 17.7/19.6/24.4 us per row. A timed A/B on the main CI wheel with the PR's files overlaid will follow before any speed claim.
Proposed change, in the PR for this issue: insert_many writes those values as tagged values inside the arcadedb_n_insert_rows text; a row with a value the text still cannot carry (NaN, Infinity, an integer beyond 64 bits, a date-time or Decimal inside a list or dict, a non-str key, a lone surrogate, a datetime.time) goes to the statement on its own while the rows around it stay on the bulk entry in the order given.
Activity