Repository navigation
Four silent failures with success exit codes: cluster-only ignores --backend, label prompt collides with the Community {cid} sentinel, tree --root <abs> flattens the hierarchy, built_at_commit stamped from cwd #2534
Description
Four independent bugs, filed together because they share one shape: the command exits 0, prints a success line, and produces output that looks right but isn't. There is no error for a user — or an agent — to react to.
All line references are against v8 as of today, so these are not stale-version reports.
1. cluster-only --backend X accepts labeling flags and never calls the LLM
cli.py:1621 sets force_relabel = cmd == "label", and cli.py:1776 gates the whole labeling block on it:
force_relabel = cmd == "label"
...
if labels_path.exists() and not force_relabel:
# reuse saved labels / hub-rename; --backend, --model, --batch-size unusedSo once .graphify_labels.json exists, cluster-only --backend openai --model … --batch-size N silently discards every one of those flags. It prints Done - N communities. GRAPH_REPORT.md, graph.json and graph.html updated., and GRAPH_REPORT.md records Token cost: 0 input · 0 output.
There is a warning at cli.py:1830, but it only fires if changed: — i.e. only when the community set drifted. When the count matches, the no-op is completely silent.
The flags being accepted is what sells it: a rejected flag would have been obvious immediately. It took two labeling runs and a look at the token counter to notice nothing had happened.
Suggested fix: if --backend / --model / --batch-size are passed explicitly to cluster-only while the reuse branch is taken, either honour them or warn naming the ignored flags. Silently accepting a flag that does nothing is the worst of the three options.
2. The label prompt collides with the placeholder sentinel, so some models' answers get discarded
llm.py:2859 formats each prompt line using the same string that llm.py:2834 uses as the "unnamed" sentinel:
lines.append(f"Community {cid}: {', '.join(names)}") # llm.py:2859
return {int(cid): f"Community {cid}" for cid in communities} # llm.py:2834Models that pattern-match the input key echo it straight back as the value — {"5": "Community 5"} — and cli.py:1874 then drops exactly that string:
labels.update({cid: v for cid, v in generated_labels.items()
if v and v != f"Community {cid}"})Net effect: the LLM answered, the answer was thrown away, and the deterministic hub name was kept. No warning, no error, no token-cost anomaly. It reads as "the model chose not to name this cluster".
Same graph, same prompt, same batch size — only the model varies:
| model | share of communities named |
|---|---|
google/gemini-3.6-flash |
~27% |
google/gemini-3.5-flash |
echoed the sentinel on half of an isolated sample |
openai/gpt-5.4-mini |
all of the same sample |
google/gemma-4-31b-it |
~100% |
The same small set of communities, isolated (labels redacted, <name> = a correct plain-language label):
gemini-3.6-flash -> {0: '<name>', 1: 'Community 1', 2: 'Community 2', 5: 'Community 5'}
gpt-5.4-mini -> {0: '<name>', 1: '<name>', 2: '<name>', 5: '<name>'}
Every cluster in that sample was plainly nameable — two other models named all of them. Gemini is echoing the prompt's key prefix, not declining to answer.
Worth noting how this compounds with the hub-label fallback: because the fallback is plausible (a real symbol name), nothing about the output looks broken. The fallback is frequently a bare filename, and a large cluster can end up named after a trivial shared helper.
Suggested fix: stop using Community {cid} as both the prompt key and the sentinel. An opaque prompt key (C0:, [0], or a bare index) removes the collision for every model at zero cost. Optionally also warn when a batch returns >50% sentinel values, which would surface this on the first run.
3. tree --root <absolute path> silently flattens the hierarchy
graph.json stores source_file repo-relative (<dir>/<subdir>/<file>). tree_html.py:115-122 does:
for src_file, syms in sorted(by_file.items()):
src_path = Path(src_file)
try:
rel = src_path.relative_to(root_path)
parent_path = (root_path / rel).parent
except ValueError:
parent_path = root_path # <-- every file lands here--root is documented as "filesystem root for the hierarchy", so an absolute path is the obvious thing to pass. When you do, relative_to raises for every file and the except attaches all of them directly to the root.
The result: max depth 2, and the root's children become the flat set of every file basename in the graph rather than the top-level directories. Siblings collide — distinct files sharing a basename appear as indistinguishable top-level entries. It still writes the HTML and reports success, with nothing to indicate the hierarchy was discarded.
Suggested fix: treat a --root that resolves zero source_file entries as an error, or normalise it against the graph's own path convention before use. Even a 0 of N files resolved under --root warning would be enough.
4. cluster-only re-stamps built_at_commit from the shell's cwd, not from the analysed path
GRAPH_REPORT.md carries a provenance line the docs invite you to trust:
## Graph Freshness
- Built from commit: `<sha>`
- Run `git rev-parse HEAD` and compare to check if the graph is stale.
extract --out <dir> stamps that correctly. But a subsequent cluster-only <dir> run from a different working directory overwrites it with the cwd's HEAD, and the path being analysed is never consulted.
Reproduced by extracting two different revisions of the same repo into two separate output dirs, then running cluster-only on both from one shell:
graph A (extracted from revision X) -> built_at_commit: <cwd HEAD>
graph B (extracted from revision Y) -> built_at_commit: <cwd HEAD> # identical
Both graphs claimed the same commit despite demonstrably different contents (different file counts, different node/edge totals). The failure is quiet and the consequence is specific: the one field designed to tell you whether a graph is stale is the field that goes wrong, and the suggested git rev-parse HEAD check actively confirms the false answer when you happen to run it from the stamping cwd.
This also makes graphs non-comparable across revisions, which is otherwise a natural use — extract two revisions, diff the graphs, measure what a refactor changed.
Suggested fix: resolve built_at_commit from the analysed path (the --out/target dir's source root), not from os.getcwd(); or leave the value written by extract untouched, since cluster-only re-clusters an existing graph rather than re-reading the source tree.
Happy to split these into four issues if that suits triage better. I grouped them because the shared "success exit, wrong output" property is arguably a more useful signal than any one of them alone.
Activity
[-]Three silent failures with success exit codes: cluster-only ignores --backend, label prompt collides with the `Community {cid}` sentinel, `tree --root <abs>` flattens the hierarchy[/-][+]Four silent failures with success exit codes: cluster-only ignores --backend, label prompt collides with the `Community {cid}` sentinel, `tree --root <abs>` flattens the hierarchy, built_at_commit stamped from cwd[/+]fix(cli): surface four silent success-exit failures (#2534)
safishamsi commentedon Aug 7, 2026
Fixed in v0.9.36 (
graphifyy==0.9.36on PyPI). All four cases now surface:cluster-onlywarns when--backend/--model/--batch-sizeare ignored on the label-reuse path; the community-label prompt key no longer collides with theCommunity <id>discard sentinel (a model echoing the key back is no longer silently dropped);tree --rootexits non-zero with a clear message when the root matches no source file instead of silently flattening the tree; andcluster-onlystampsbuilt_at_commitfrom the analysed graph rather than the shell working directory. Thanks for the precise list.Merge upstream/v8 (v0.9.37) into chore/sync-upstream-0937
fix(cli): surface four silent success-exit failures (Graphify-Labs#2534)
fix(export,cli): stamp graph provenance from the analysed repo, not t…
fix(tree): one root per repo on merged graphs; refuse graphs with no …