claude-cli backend returns an agentic summary instead of JSON on Claude Code 2.1.216 (the llm.py:1376 workaround has regressed) #2076

The claude-cli backend produces an empty graph on Claude Code 2.1.216. The model does the extraction and then reports it conversationally, so _response_is_hollow reads truncation, bisects, and converges on nothing.

Repro

graphify extract ./corpus --backend claude-cli --out ./out \
  --max-concurrency 1 --token-budget 12000 --api-timeout 900

Two markdown documents. Output:

[graphify] LLM returned invalid JSON, skipping chunk (first 200 chars:
  Knowledge graph extracted and delivered — 21 nodes, 20 edges, 3 hyperedges
  from the Codex↔NIM probe document.)
[graphify] claude-cli returned a hollow response; treating as truncation so
  adaptive retry can bisect the chunk.
[graphify] single-file chunk ... truncated at max_completion_tokens
[graphify extract] graph is empty — extraction produced no nodes.
rc=1

The two semantic cache entries written are {"partial": true, "nodes": [], "edges": []}.

Note the response text: the extraction succeeded — 21 nodes / 20 edges / 3 hyperedges — and was then summarised rather than returned. The work is done and discarded.

This is the failure llm.py:1376-1391 already describes

That comment documents exactly this class, and states the fix:

Putting the full extraction schema plus an explicit imperative in the user turn — and dropping --system-prompt — makes the CLI emit the JSON object directly. […] verified against Claude Code 2.1.197

We run 2.1.216. The workaround appears to have regressed somewhere in between.

Isolation — four arms

So the transport is fine and the CLI is fine; it is the prompt shape that now reads as an agentic task rather than a formatting instruction.

Suggestions

• Pin the version this is verified against in CI, or probe it at runtime — the comment already shows this workaround is version-sensitive, and it silently produces an empty graph rather than an error when it breaks.
• Consider treating a successful-sounding prose response distinctly from a truncated one. Right now both route to the bisect path, which cannot converge when the model is answering rather than truncating, so it burns retries before failing.

Environment: graphify 0.9.22 (pipx), Claude Code 2.1.216, macOS arm64.