Skip to content

graphify claude install (v0.8.41): 26 validated bugs in the generated Claude Code skill (interpreter, --directed, shrink-guard, cleanup ordering) #1392

Closed

Description

@illiakaliuzhnyi-preply

Bug bundle: graphify claude install skill (v0.8.41) — 26 validated issues in the generated Claude Code skill

The auto-generated Claude Code skill produced by graphify claude install (pinned .graphify_version = 0.8.41) ships several reproducible bugs in SKILL.md and the references/*.md runbooks. We vendored this snapshot into a monorepo and an automated reviewer (plus manual checks) surfaced the issues below. Each was verified against the installed graphify v0.8.41 Python source (build.py, export.py, etc.), not just read off the markdown.

Reporting upstream because the skill is regenerated by graphify claude install, so any downstream edit to the vendored copy is overwritten on the next install/upgrade — the fix has to live in the generator.

Happy to open individual issues or send a PR against the skill template if that's easier; grouping here so the pattern is visible.

A. Interpreter resolution — generated commands call system python3, which can't import graphify under uv tool install

  1. SKILL.md interpreter bootstrap — uv tool run graphifyy python selects no package and falls through to system python3. The skill should resolve and persist the tool's interpreter (e.g. read the shebang of the installed graphify bin) — we use $(cat graphify-out/.graphify_python).
  2. references/add-watch.md --watch — runs bare python3 -m graphify.watch; not importable under uv tool install. Must use the resolved interpreter.
  3. references/exports.md --mcp — python3 -m graphify.serve and the generated Claude Desktop config command both point at system python; not importable. Must use the resolved interpreter / its absolute path.

B. Environment variables set but never exported — child Python reads os.environ and never sees the override

  1. references/transcribe.md — GRAPHIFY_WHISPER_MODEL=base set as an unexported shell var; transcriber silently uses the default model.
  2. references/transcribe.md — Step 1 sets WHISPER_PROMPT but the transcriber reads GRAPHIFY_WHISPER_PROMPT; domain hint silently ignored (name mismatch + missing export).

C. --directed not propagated — directed builds silently produce undirected graphs

  1. SKILL.md Steps 4/5 — build_from_json(extraction) omits directed=, so --directed builds undirected.
  2. references/update.md merge — the update-only merge calls build_merge(...) without directed=; build_merge(..., directed: bool = False) (build.py:448) then builds undirected via build(all_chunks, directed=directed). On --directed --update, reciprocal A↔B edges collapse to one when the merged graph is written back through G.edges(), and the later directed rebuild cannot recover the dropped edge.

D. Output path uses scan-root, but later steps read cwd graphify-out/ (or vice-versa)

  1. SKILL.md (CHUNK_PATH) — chunk output path is derived from .graphify_root (the scanned dir) but Step B3 merges from cwd graphify-out/; a non-cwd scan (graphify <subdir>) writes chunks where the merge never looks → "no nodes".
  2. references/add-watch.md ingest — /graphify add <url> ingests into cwd Path('./raw'), but the following --update scans $(cat .graphify_root); a non-cwd build saves the file outside the scanned root → "no new files". Ingest into root/'raw'.

E. File-list / category scoping

  1. SKILL.md Step B0 — all_files flattens every detect['files'] category, including code, into the semantic-extraction list. code is already covered structurally by the AST pass, so LLM subagents re-read every source file — a repo-root run with even one README becomes unexpectedly expensive. Scope to content categories only ({document, paper, image}; video → document via transcription).
  2. references/update.md / transcribe.md — on --update, the non-code branch jumps straight to extraction without running transcription, so changed .mp4/.mp3 stay in files.video and feed unreadable media paths into semantic extraction (and a video-only corpus hits the code-only fast path → empty graph). Route media through transcription, then rewrite .graphify_detect.json to move transcripts into files.document and drop files.video.
  3. SKILL.md ~230 (stale semantic cache) — .graphify_cached.json is only (re)written when check_semantic_cache returns hits (if cached_nodes or ...:). If a prior run took the all-cached shortcut and skipped Part B cleanup, the file is left behind; on a later run with zero cache hits but some uncached files the conditional is false, the stale file survives, and Step B3 merges its unrelated nodes/edges into the new graph. Always rewrite (empty) or delete the cache file before the check.

F. Cleanup deletes intermediate files that later steps still read

  1. SKILL.md / code-only path — code-only runs skip Part C but the merge reads .graphify_semantic.json / .graphify_extract.json → FileNotFoundError. Write an empty semantic file and run Part C so the merge input exists.
  2. references/update.md --cluster-only — instructs "run Steps 5–9 as normal", but a prior build's Step 9 deletes .graphify_{extract,detect,analysis}.json that Step 5 reads → FileNotFoundError. The graphify cluster-only CLI is self-contained (re-clusters, regenerates outputs) — rely on it, don't re-run Steps 5–9.
  3. references/update.md update tail — "run Steps 4–9 as normal" then "after Step 4 show the diff", but Step 9 cleanup deletes .graphify_extract.json which the diff block reads → every successful update fails at the diff. Order must be Steps 4–8 → diff (+ rm .graphify_old.json) → Step 9 last.

G. graph.json overwrite / shrink-guard safety (#479)

  1. references/update.md prune — prune was built from deleted + changed. build_merge already drops every re-extracted source_file from the base before merging (re-extracted files replace their prior contribution), then applies prune_sources to the final merged graph — so passing changed files in prune deletes the freshly re-extracted nodes, leaving graph.json missing that file. prune should be deleted-files-only.
  2. SKILL.md Step 4 — the to_json(...) return value was ignored; it returns False and writes nothing when the new graph is smaller than the existing graph.json (#479 shrink-guard, export.py). A deletion --update then saved the manifest/report over a stale graph.json. Capture the return; abort loudly on refusal.
  3. SKILL.md Step 4 (zero-node guard) — the empty-extraction guard ran after GRAPH_REPORT.md, to_json(force=...), and .graphify_analysis.json were written, so an empty extraction on --update (with force=True) overwrote a good graph.json before aborting. Move the G.number_of_nodes() == 0 check to immediately after build_from_json(...), before any write.
  4. SKILL.md is_update — force was decided by the mere presence of .graphify_old.json. An update that aborts after creating the backup but before cleanup leaves the file behind, so a later plain rebuild reads it, passes force=True, and bypasses the shrink-guard. Decide force from an explicit substituted IS_UPDATE flag instead (fail-safe: guard stays on if unset).
  5. SKILL.md ~438 (report before export) — GRAPH_REPORT.md is written before the to_json shrink-guard runs. On a non---update rebuild that produces fewer nodes, to_json returns False and aborts without replacing graph.json, but the report has already been overwritten — leaving the report describing a graph that queries (still on the old graph.json) never see. Write the report only after a successful export (or stage it in a temp file).

H. Misc

  1. references/extraction-spec.md — schema example uses a forbidden filename-only node ID (session_validatetoken), which the spec itself warns creates ghost/duplicate nodes. Use the immediate-parent+filename form (auth_session_validatetoken).
  2. references/query.md fallback traversal — token filter len(t) > 3 drops valid 3-letter vocabulary tokens (api, jwt, ios); the vocab threshold is >= 3. Align to len(t) >= 3.
  3. references/transcribe.md — the transcription block redirects all Python stdout to .graphify_transcripts.json via shell > ...json, but transcribe_all/download_audio print progress to stdout → polluted JSON → the next json.loads fails. Write the file from Python (Path.write_text(json.dumps(...))), send status to stderr.

Environment: graphifyy v0.8.41 (PyPI), installed via uv tool install graphifyy; skill generated by graphify claude install --platform claude. All line references are to the generated v0.8.41 skill files.

Activity

  1. added 2 commits that reference this issue on Jun 19, 2026
  2. safishamsi commented on Jun 19, 2026

    @safishamsi
    Member

    Update: the remaining non-crash bugs are now fixed on v8 (382b669), following the crash/data-loss tier (30fe8ba).

    This batch (Claude-path skill fragments):

    Triage of the 23 numbered items: 17 real (now all fixed), 2 already fixed via #1366 (#15/#16), 4 not-a-bug (#1/#9/#17/#19 — covered by the existing "replace python3 with the resolved interpreter" instruction).

    Note on Aider/Devin: those monoliths are pinned by a roundtrip invariant and were intentionally excluded here. Their own Step 1 already instructs replacing python3 with the resolved interpreter, so the interpreter items don't apply; the other items there are tracked as a separate follow-up that bumps the roundtrip reference. Will ship in the next release.

  3. safishamsi commented on Jun 19, 2026

    @safishamsi
    Member

    All actionable items are fixed and released in 0.8.44 (live on PyPI). Upgrade with uv tool install --upgrade graphifyy (or pip install -U graphifyy), then graphify install to refresh the skill.

    Final triage of the 23 numbered items: 17 real (all fixed), 2 already fixed, 4 not-a-bug.

    Crash/data-loss tier (commit 30fe8ba):

    • cwd-relative chunk paths so a non-cwd scan no longer yields "no nodes"
    • code-only corpora write an empty .graphify_semantic.json before Part C (no more FileNotFoundError)
    • --cluster-only uses the self-contained graphify cluster-only CLI instead of re-running steps that read deleted intermediates
    • zero-node guard runs before any write; report/analysis written only when to_json persisted the graph (respects the Prevent partial-chunk overwrite in --update: pre-build node-count assertion + build_merge() helper #479 shrink-guard)

    Remaining correctness tier (commit 382b669):

    • --directed propagated into build_from_json (build + rebuild) and build_merge (--update merge + diff), so reciprocal edges are no longer collapsed
    • semantic extraction scoped to document/paper/image (code covered by the AST pass)
    • .graphify_cached.json deleted on cache miss so stale caches are never merged
    • --update transcribes changed video into documents before the semantic pass
    • transcription writes via write_text, honours GRAPHIFY_WHISPER_MODEL/GRAPHIFY_WHISPER_PROMPT, status to stderr
    • add-watch/exports use the resolved interpreter explicitly; MCP Desktop config documents the absolute interpreter path
    • extraction-spec example id namespaced; query term split keeps tokens of 3+ chars

    Already fixed before this batch (via #1366, shipped 0.8.42): #15, #16.

    Not-a-bug (the skill's Step 1 already instructs replacing python3 with the resolved interpreter $(cat graphify-out/.graphify_python), which covers these): #1, #9, #17, #19.

    Each fix was smoke-tested locally against the real library/CLI (directed reciprocal-edge preservation, the to_json shrink-guard refusing to clobber a larger graph, content-only file scoping, cache unlink-on-miss, and cluster-only regenerating the report with all intermediates deleted), in addition to the full unit suite (2210 passed).

    Note: the Aider and Devin monolith skills are pinned by a roundtrip invariant and were intentionally excluded from the runbook edits; their own Step 1 already resolves the interpreter, so the interpreter items don't apply there. Any monolith-specific follow-up will bump the roundtrip reference separately.

    Thanks for the unusually thorough report. Closing as resolved; please reopen if anything in 0.8.44 still misbehaves.

  4. safishamsi commented on Jun 19, 2026

    @safishamsi
    Member

    Follow-up: the Aider and Devin monolith skills now carry these fixes too (commit applied on v8, will ship in the next release).

    Those two are hand-maintained single-file skills, frozen against a pinned pristine-v8 blob by a round-trip guard, so they were excluded from the first pass. I evolved the guard from a positional, line-count-exact check into a multiset diff that classifies every added/removed line against documented sanctioned change-classes - so the multi-line fixes can land while any unsanctioned drift still fails the test.

    Both monoliths now: thread directed=IS_DIRECTED through every build_from_json call (a --directed run no longer collapses reciprocal edges), scope semantic extraction to document/paper/image, unlink a stale .graphify_cached.json on a cache miss, and run Step 4's zero-node guard before any write with the report/analysis gated on to_json persisting the graph.

    Verified by extracting the rendered monolith's Step 4 block and running it: an empty extraction now fires the guard before any write and leaves a good existing graph and report untouched. That closes the last gap noted in the previous comment (the monolith exclusion).

  5. added a commit that references this issue on Jun 19, 2026
  6. fenghaitao commented on Jun 24, 2026

    @fenghaitao

    On E.10: SKILL.md Step B0 — all_files flattens every detect['files'] category, including code, into the semantic-extraction list. code is already covered structurally by the AST pass, so LLM subagents re-read every source file — a repo-root run with even one README becomes unexpectedly expensive. Scope to content categories only ({document, paper, image}; video → document via transcription).

    This should bring a regression. We have the cross doc<->code edges before this change, and then after this, those edges are gone. When we give the code and doc together and ask LLM to extract, we will create entities from the code, and those entities could be merged to the AST if their ids are identifcal, after that, the edges from the doc will be pointed to the AST node.

    Should we give the code index.md and then update the prompt so that we could get those edges back?

    And after this change, are the code session in graphify/skills/claude/references/extraction-spec.md dead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions