Skip to content

algo.hashgnn's MinHash phase is O(nodeCount × embeddingDimension²) and sits outside every checkpoint (follow-up to #6264) #6295

Description

@lvca

Found while implementing #6264 (PR #6288), verified against main at 6b20fb1.

#6216 and #6264 between them gave every iteration-shaped knob in algo.* a domain minimum and a checkpoint inside the loop it drives. algo.hashgnn got both. It is still not abortable, because its dominant phase is not inside that loop.

Measured

arcadedb.command.timeout set to 1000 ms, on a 2000-node graph of degree 4:

CALL algo.hashgnn({embeddingDimension: 4096, iterations: 1, seed: 1}) YIELD node RETURN node

Ran for 112,988 ms and returned a result. The deadline was overshot by 113x and never observed — the call did not abort, it succeeded.

Why the #6216/#6264 lens walked past it

Both issues hunted knobs whose value is unbounded. embeddingDimension is not one: #6065/#6214 capped it at MAX_EMBEDDING_DIMENSION (4096), so it looks handled. The trap is that the MinHash reduction runs after the iterations loop, and costs, per node:

// AlgoHashGNN.java, after the guarded loop
for (int i = 0; i < n; i++) {              // <- no checkpoint
  for (int d = 0; d < embDim; d++) {
    for (int f = 0; f < numFeatures; f++) { ... }   // numFeatures = max(embDim * 4, 64)

embDim × numFeatures = 4 × embeddingDimension² = 6.7e7 at the cap. That per-node figure is bounded — and then multiplied by an unbounded nodeCount. A bounded per-item cost times an unbounded item count is unbounded work, and here it is the dominant phase, not a tail: at 2000 nodes it is 1.3e11 inner iterations, which is the whole 113 seconds.

So the shape to look for is not "which knob has no ceiling" but "which loop over nodeCount has no checkpoint", and the phases after the knob-driven loop are exactly where #6264 stopped looking.

Fix

One line: guard.checkPeriodically(i) at the head of that loop. The WorkGuard already exists in the method (#6264 added it); the reduction simply sits outside its reach. Per-node work there is 6.7e7 operations at the cap, so the throttled form is more than fine — even the unthrottled check() would be free.

Second instance, same shape, much smaller

AlgoSLPA's post-processing loop (freq.merge(...) per remembered label, boxed) is also outside the guarded region and is O(nodeCount × iterations). It is indirectly bounded, because #6264 prices nodeCount × iterations against arcadedb.cypher.algoMaxWalkMemory — at the 64 MB default that caps it near 1.6e7 boxed merges (~1 s). It scales with the budget, though, so an operator who raises the budget re-opens it. Measured at degree 20000: 1086 ms against a 1000 ms deadline, i.e. real but nowhere near HashGNN's 113x.

Worth doing the sweep across the package rather than patching these two: the question for each procedure is whether every loop over nodeCount is covered, not just the one the knob drives.

Not this issue

Related

#6216, #6264, PR #6288, PR #6222, #6263/PR #6285, #6266/PR #6291.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions