Part B2 semantic-extraction subagents silently crash past the ~64k output-token turn limit on large/deep-mode chunks; SKILL.md chunking is file-count-based, not output-size-aware #1758

Activity

1. DataMindBI commented on Jul 16, 2026

   

   DataMindBI

   on Jul 16, 2026

   More actions

   Confirming this independently — hit the identical failure mode running /graphify (Agent-tool subagent dispatch, no Gemini key configured) over a ~370-file corpus split into 17 semantic-extraction chunks at the default Step B1 sizing (20-25 files/chunk):

   • 6 of 17 chunks (~35%) failed outright.
   • 5 failed with API Error: Claude's response exceeded the 64000 output token maximum.
   • 1 failed with a generic mid-response server error.
   • Failures clustered specifically on chunks containing dense, lengthy documents (formal filings, long-form correspondence, detailed multi-page responses) rather than chunks made up of shorter documents (e.g. routine emails) — even though both chunk types had the same file count. This lines up with the root cause described here: flat file-count chunking doesn't account for per-file content density, so two 20-25-file chunks can have very different expected output volume.

   +1 on both suggested fixes:

   1. Size chunks dynamically by estimated content volume/word count rather than a flat file count.
   2. Detect the 64000-output-token failure mode specifically and auto-retry that chunk at half size instead of silently skipping it — right now a failed chunk is silent data loss unless the user notices the "N/17 succeeded" count and manually re-runs.

   Happy to share more detail on the corpus profile if useful for reproducing.
2. Abdallah01 commented on Jul 27, 2026

   

   Abdallah01

   on Jul 27, 2026

   Last edited by Abdallah01

   More actions

   Independent corroboration from a third corpus: this still reproduces on 0.9.27
   (this report was filed against 0.8.36), and one detail in the original write-up
   looks too narrow.

   --mode deep is not required. The report says deep mode "is the trigger, not an
   edge case", which reads as though deep mode is needed to hit the ceiling. Our run
   used default mode, no --mode deep, and still lost two chunks to
   response exceeded the 64000 output token maximum. So the ceiling is reachable on a
   plain /graphify . — deep mode raises the odds, it isn't a precondition.

   Density, not file count — measured. 10 chunks dispatched at default Step B1
   sizing. The two that died were the two densest, and nothing else:

   chunk	files	content	outcome
   04	21	archived design documents	died on the 64k cap
   07	21	architecture reference documents	died on the 64k cap
   08	19	build/config files	fine — emitted 47 nodes

   Same file count, roughly 10x the output. That lines up with the independent
   confirmation above from a very different corpus profile, which I think strengthens
   the case that the problem is the metric rather than any particular kind of
   document.

   Thirds worked where halves might not have. The suggested fix upthread is
   auto-retry at half size. Worth noting we needed thirds: re-splitting each
   failed 21-file chunk into three 7-file sub-chunks succeeded first try, both times.
   A halved 21-file chunk would still have been ~10 dense files, and given how far over
   the cap they were, I doubt that would have cleared. If auto-retry lands, a loop that
   keeps subdividing until it fits would be safer than a single halving — and the retry
   should be on the same chunk re-split, never the chunk re-sent unchanged, which
   just burns the budget again.

   Two smaller things that helped and are cheap to put in the prompt:

   • Ask for compact JSON. Non-pretty-printed output means the cap measures content
     instead of whitespace. On dense chunks that is not a rounding error.
   • Give the subagent its own node/edge bound, so it degrades instead of dying.
     A chunk that returns 60 good nodes is recoverable; a chunk that dies returns
     nothing and its files stay uncached for the whole run.

   The Step B3 misdiagnosis is still present in 0.9.27. A missing chunk file is
   still explained as "subagent may have been read-only. Re-run with general-purpose
   agent." Both of our failures used general-purpose correctly, so that message sent
   us looking at subagent_type first. Since the real cause is now known, it would help
   if that branch listed the output-ceiling crash as the other cause of a missing chunk
   file — one line, and it saves the wrong investigation.

   Environment: graphifyy 0.9.27, Windows 11, Agent-tool subagent dispatch with no
   Gemini key set, subagent_type="general-purpose", default mode, mixed corpus of
   source plus project documentation (~210 non-code files dispatched, 62 images
   deliberately skipped).

   Separately filed #2229 for a different gap this same run surfaced — the #479
   shrink-guard compares total node count, so the total collapse of a semantic layer
   that this issue describes would not be caught by it if the AST layer happened to
   grow in the same run.
3. maxhawk1 commented on Aug 7, 2026

   

   maxhawk1

   on Aug 7, 2026

   More actions

   Fourth corpus confirming this, still present on 0.9.20. Adding three things that aren't yet in this thread: an implementation of suggestion #3 that does not degrade to partial output, a distinct second failure mode on the same Write call, and a gotcha for whoever implements it.

   (Same disclosure as the original report: investigation and write-up done with an AI assistant, based on a real run.)

   1. Suggestion #3 works, and it doesn't have to be lossy

   The write-up frames incremental writes as "more invasive, but it turns a hard failure into a partial one." In practice it turns it into a complete one. The batching goes on the write path, not the extraction — the model still extracts the whole chunk, it just doesn't have to emit it in a single turn's output budget.

   Protocol we ran (verified end-to-end, including the failure paths):

   1. Init a sidecar: mkdir -p "$(dirname CHUNK_PATH)" && : > 'CHUNK_PATH.parts.jsonl'
   2. Append batches of ~40 nodes / ~60 edges per call, one JSON object per line, each validated as it lands so a malformed batch fails loudly instead of corrupting the file:

      python3 -c 'import json,sys; d=json.load(sys.stdin); open("CHUNK_PATH.parts.jsonl","a").write(json.dumps(d)+"\n")' <<'BATCH_JSON'
      {"nodes":[...],"edges":[...],"hyperedges":[]}
      BATCH_JSON
      

      The quoted heredoc delimiter matters — without it the shell eats $, backticks and backslashes inside labels.
   3. Assemble with python3, deduping nodes by id and edges by (source, target, relation), so overlapping batches are safe.
   4. Remove the sidecar.

   Three properties that matter: no single turn approaches the ceiling; python3 writes the bytes so no tooling sits between the model and the file; and a failure costs one batch rather than the chunk. Re-dispatching our three failed chunks under exactly this protocol recovered all three.

   We paired it with explicit caps in the extraction prompt (150 nodes / 250 edges / 3 hyperedges, "truncated": true when the source genuinely exceeds them) and a hard cap on the subagent's final message. Worth noting the prompt currently opens with "Output ONLY valid JSON matching the schema below" and separately instructs a Write to disk — that ambiguity invites the model to emit the payload into its output budget and write it, which burns the ceiling twice over. Making it unambiguous that the JSON only ever travels through tool calls is a cheap independent win.

   2. Distinct failure mode on the same call: the Write auto-formatter truncates valid output

   Separate from the token ceiling, we observed Write's auto-formatter force-closing an intentionally-open JSON array, producing a file that exists and parses but is silently missing most of its content. Same chunk, same run:

   file	size
   .graphify_chunk_05.json.truncated (formatter-mangled)	12,315 B
   .graphify_chunk_05.json (valid re-run)	45,816 B

   This is a different symptom from the one this issue describes — the file is written, so Step B3's existence check passes and the short chunk merges cleanly. Nothing downstream flags it. It's arguably worse than the crash for that reason, and it's a second independent argument that Write is the wrong primitive for this payload regardless of chunk sizing.

   If Step B3 keeps an existence check, it may be worth also treating "file is short relative to sibling chunks and ends mid-structure" as a truncated write rather than a small chunk.

   3. Implementation gotcha: sidecar naming vs the Part C merge glob

   Part C globs graphify-out/.graphify_chunk_*.json. Name the sidecar CHUNK_PATH + ".parts.jsonl" and it's safe — .graphify_chunk_04.json.parts.jsonl does not match that pattern (the pattern requires the name to end in .json). We verified this explicitly, because getting it wrong would feed partial batch files straight into the merge.

   4. Loss rate

   19 chunks over dense primary-source documents (SEC filings, regulatory dockets, testimony PDFs); chunks 04, 06 and 14 lost = 16%. Consistent with @DataMindBI's ~35% on a denser corpus. The cost profile is worth stating plainly: it dies after all file reading completes, so the full token cost of extraction is paid and nothing is written.

   One small confirmation of the write-up's point about the misleading Step B3 diagnostic — chunk 04's last line before it died was:

   I have now finished reading all 22 files and have a complete extraction ready. Let me write the knowledge-graph fragment to the required output path.

   Nothing about that looks like a read-only dispatch.

   Scope

   The Write-tool instruction ships in 10 references/extraction-spec.md copies across the packaged platform variants, so any fix here wants applying across all of them, not just the Claude skill.
4. omsenjalia

   added a commit that references this issue on Oct 21, 2026

   Merge upstream v8 into fix/top-impact-issues

   ...

   19e2c31