OpenAI backend caps output at 8192 tokens (half the 16384 other backends use) → truncated semantic chunks on dense repos #1372

Summary

The openai backend has no max_tokens key in its BACKENDS config (llm.py), so it falls through _resolve_max_tokens(cfg.get("max_tokens", 8192)) to 8192 — while every other backend (claude, gemini, kimi, deepseek, bedrock) is set to 16384. On dense repos/chunks the model's JSON output hits this ~8192-token (~32k-char) ceiling mid-generation (finish_reason == "length"), producing LLM returned invalid JSON … Unterminated string starting at … char ~32000, forcing the chunk-split-retry path. Inferred edges in the truncated tail are lost.

Repro

graphify extract <large-code-repo> --backend openai --token-budget 20000

→ LLM returned invalid JSON, skipping chunk: Unterminated string starting at: line 1 column ~32000
→ chunk of N truncated at depth 0, splitting into halves …

Truncation char positions cluster at 29k–36k chars (≈ 7–9k output tokens — just over 8192).

Impact

• Suppressed INFERRED/semantic edges on big repos (measured: an 8.9k-node repo gained +67 INFERRED edges once the cap was raised; 4 truncation events → 0).
• Wasted tokens on truncated-then-retried chunks.
• Silent inconsistency vs other backends — openai users get half the output headroom, no signal.

Workaround (works today)

export GRAPHIFY_MAX_OUTPUT_TOKENS=16384 restores parity. Bonus: truncated chunks aren't cached (they errored), so a --force re-run only re-does the failed chunks — recovery is cheap.

Suggested fix (RFC — pick the altitude)

1. Parity: add "max_tokens": 16384 to the openai backend config (one line; removes the surprise).
2. Output-aware: chunking is sized by input (--token-budget) but the cap is about output. On finish_reason == "length" (already detected), auto-retry the chunk once at a higher cap before falling back to split — or size chunks by estimated output (≈ f(#entities)).
3. Per-model introspection: derive the cap from the model's actual max_completion_tokens capability rather than a hardcoded default.

Happy to PR option 1. The env override is a fine interim; the ask is (a) fix the default for parity, (b) consider output-aware chunking so dense repos don't silently lose edges.