Question — graphify update non-determinism produces ~11k-line diff per run on unchanged source: intended or addressable? #741

Observation

graphify update <path> is non-idempotent on a clean tree. Running it back-to-back with no source changes regenerates graph.json + GRAPH_REPORT.md with ~11k-line diffs, dominated by community-ID renumbers.

Sample diff from a 12,418-node graph (graphifyy 0.7.5), no intervening source changes between runs:

graph.json:
-      "community": 1580,
+      "community": 1587,
-      "community": 1581,
+      "community": 1588,
-      "community": 1582,
+      "community": 1589,
... (~11,800 such renumbers)

GRAPH_REPORT.md:
- 12418 nodes · 12706 edges · 3596 communities (2946 shown, 650 thin omitted)
+ 12418 nodes · 12706 edges · 3603 communities (2952 shown, 651 thin omitted)
- Built from commit: `3bbf35c9`
+ Built from commit: `66343027`

Node + edge counts identical. The shift comes from clustering.

Root cause (from upstream source — graphify/watch.py:_rebuild_code)

Each graphify update invocation:

1. Re-detects files (manifest-aware, fast)
2. Re-extracts AST for all detected files (cache-aware, fast)
3. Preserves semantic nodes from existing graph
4. Always re-runs cluster() (Leiden) on the merged graph — this step is non-deterministic enough across processes to renumber communities even when the graph topology is unchanged
5. Re-writes graph.json + GRAPH_REPORT.md unconditionally

Step 4 is the source of churn. Even with seeded Leiden (advertised "deterministic community IDs" since 0.7.0), per-process variation produces meaningful renumbers.

Practical impact

We wired a post-commit git hook to call graphify update, hoping for incremental refresh. In practice:

• Every commit produces an 11k-line diff that has no signal
• Amending a commit re-fires the hook → another 11k-line diff appended to the cascade
• PR diffs become unreviewable (signal buried under renumber noise)
• We had to disable the hook entirely

Questions

1. Is the per-process Leiden churn intended, or is the seeded-Leiden determinism guarantee narrower than expected? (e.g. only deterministic within one process, not across runs)
2. Would you accept a --no-cluster flag on graphify update that preserves existing community assignments when AST changes are localized? (The semantic-preserving merge logic in _rebuild_code lines 80-103 already preserves nodes; analogous preservation of community IDs would close the loop.)
3. Alternatively, is there an existing fast-path that skips the full rebuild when the manifest hash + extracted AST IDs match the prior run? If not, would you accept one?
4. Independent of clustering: is there a reason GRAPH_REPORT.md updates the "Built from commit" hash even when graph contents are unchanged? A no-op shortcut would help.

Happy to open a PR for any of the above if intent is "addressable, fix accepted."

Reproduction

# In any indexed graphify project after a successful initial run:
graphify update .
git diff --stat graphify-out/graph.json graphify-out/GRAPH_REPORT.md  # clean
graphify update .
git diff --stat graphify-out/graph.json graphify-out/GRAPH_REPORT.md  # ~11k lines diff

Environment

• graphifyy 0.7.5
• Python 3.13
• Project layout: knowledge/graphify/.venv + graphify-out/ colocated under a project subdir
• Graph size: ~12,400 nodes / ~12,700 edges / ~3,600 communities (semantic-augmented)