Skill Part C never stamps _origin, so graphify update deletes the semantic layer of any doc without section numbers

Summary

After a semantic extraction via the skill's Part B/C (Agent-tool path, no API key), the first graphify update deletes the entire semantic layer of any document whose extracted nodes carry a line-number source_location. Measured on a 21-document run: 8 of 21 files lost, 133 nodes and 236 edges, on a commit that touched only one of those files.

Version: graphify 0.9.46, skill-windows.md variant, Windows 11 / Python 3.10.

Root cause

build._is_ast_tier decides a node's tier from _origin and falls back to shape when it is absent:

def _is_ast_tier(item: dict) -> bool:
    o = item.get("_origin")
    if o is not None:
        return o == "ast"
    loc = item.get("source_location")
    return isinstance(loc, str) and bool(_AST_LOC_RE.match(loc))   # 'L<line>' -> AST

SKILL.md Part C merges AST + semantic and never sets _origin on the semantic side. So a semantic node's tier is decided by what its source_location happens to look like, which in turn is decided by whether the source document uses section numbers:

document style	agent writes	`_is_ast_tier`	outcome
has section marks	`"§4"`, `"§5.2"`	False	recognised as semantic, preserved
no sections	`"L8-L12"`, `"L170-179"`	True	treated as an AST heading node

In watch._rebuild_code, a file with no surviving semantic-tier node never enters semantic_doc_identities, so the #1915 guard does not apply to it, it is AST-quick-scanned, and its semantic nodes are evicted as AST-tier output of a re-extracted source.

Measured on our corpus: 135 of 409 semantic nodes are misread by the shape fallback.

Why nothing catches it

Three independent safety nets are all inapplicable on this exact path:

_rebuild_code calls to_json(..., force=True), so the Prevent partial-chunk overwrite in --update: pre-build node-count assertion + build_merge() helper #479 shrink guard cannot refuse the smaller graph.
graphify update never consults cache/semantic/, so nothing re-derives the dropped nodes even though they are still on disk.
The manifest still reports the files as semantically extracted, so detect_incremental will not re-queue them either.

Net effect: nodes gone, cache intact, every surface green. It is only visible by counting _origin: "semantic" nodes per file — a corpus-wide total still looks healthy.

Reproduce

Semantic-extract two markdown docs via the skill's Part B: one using section-numbered headings, one using none.
Complete Part C and Step 4 as written.
graphify update .
Count _origin-less nodes per source file. The unsectioned document's semantic nodes are gone; the sectioned one's survive.

Fix

One line each in SKILL.md Part C:

for n in sem['nodes']:
    n['_origin'] = 'semantic'          # add
    ...
for e in sem['edges']:                 # add
    e['_origin'] = 'semantic'

_is_ast_tier then short-circuits on the marker and never reaches the shape fallback. Verified across four subsequent commits: 21/21 files retained their layer, 385 nodes and 571 edges unchanged.

A defence-in-depth option worth considering separately: have build_from_json stamp tier at build time rather than relying on every producer to do it, since the shape fallback is load-bearing for legacy graphs and will keep catching new producers.

Companion issue from the same run: #2844 — Step 9 passes the AST+semantic merge to _stamped_manifest_files. Same shape: a provenance/tier signal that Part C does not carry through.

Another corpus hit this, but through a different door — worth recording since it narrows what the fix has to cover.

The shape fallback is not what bit us. Our graph carries 2,060 semantic nodes with no _origin, and zero of them would be misread by _is_ast_tier: 1,894 have an empty source_location and the rest a ## heading string, never L<line>. So the tier detection this issue fixes was, for us, working.

We lost the layer anyway, because of the third bullet in your "why nothing catches it" list. _rebuild_code reaches to_json(..., force=True) (watch.py:1768 in 0.9.50), so the #479 guard is bypassed unconditionally — no tier detection required. Six hooked rebuilds on 2026-08-12 landed at 2,167–2,331 nodes against a curated 6,166 (merged code+doc: 4,106 AST from 174 .py, 2,060 semantic from 271 .md); a live hook install on 2026-08-26 began the same collapse on the first commit and was killed mid-run.

So _origin stamping fixes the subtler half, and the force=True on the rebuild path remains a second, independent route to the same loss. Both are worth closing.

What we did instead of waiting, in case it helps anyone landing here: GRAPHIFY_OUT is enough to make this structurally safe with no patch, since graphify.paths resolves it at import and watch.py builds out = watch_path / _GRAPHIFY_OUT. Curated merged graph stays in graphify-out/ and is never rebuilt automatically; every automatic rebuild runs with GRAPHIFY_OUT=graphify-out-code against a disposable AST-only graph. Measured across four pinned rebuilds, incremental and full, the curated graph stayed byte-identical.

One trap on that path, detailed in #2229: a full pinned rebuild still pulls markdown in, because _rebuild_code appends AST-extractable documents. It needs a per-output .graphify_build.json exclude scope, not a .graphifyignore entry, since both graphs share a scan root.

Same root cause, opposite symptom, different code path — and still present in 0.9.50.

This issue traces the shape fallback to nodes being deleted via watch._rebuild_code. On the build_merge incremental path the same misclassification does the reverse: it strands the old semantic generation forever, so two generations of the same document accumulate in the graph.

Mechanism

build_merge's replace-on-re-extract is tier-scoped. New-chunk items are classified with _is_ast_tier (build.py:1702 in 0.9.50):

tier_sources = new_ast_sources if _is_ast_tier(n) else new_sem_sources

Unstamped semantic nodes whose source_location looks like L<line> land in new_ast_sources. The _kept() filter below then spares every prior semantic node for that source, because only the AST tier was claimed. The file is never replaced; it is merely added to.

The console line is misleading here — Replaced 1006 node(s) from re-extracted source file(s) counted only AST nodes, while every re-extracted .md kept its previous nodes.

Observed

Real corpus, graphify update . over 53 commits (131 changed files, 10 re-extracted docs), 0.9.46:

source_file	nodes after update	nodes actually extracted this run
`docs/despliegue.md`	29	9
`CLAUDE.md`	24	23

The 20 extra nodes on docs/despliegue.md were the previous run's generation, seven days stale. One of them still asserted Ops: Postgres backups (local copy only, not offsite yet) — a claim the document had already been edited to contradict. Its source_location values were L460-L565 and L13,L26-L32; the unanchored ^L\d matches both.

Worth noting for triage: the graph-health diagnostic cannot see this. No dangling endpoints, no missing endpoints, no collapsed edges — the stale nodes carry perfectly valid edges. It is only visible by comparing per-file node counts against cache/semantic/.

Still present in 0.9.50

_AST_LOC_RE is unchanged (build.py:41), and all five tier decisions still route through _is_ast_tier.

Suggested library-side fix

The Part C stamp proposed above fixes both symptoms and is the right primary fix. This is the defence-in-depth option the issue already flags as worth considering separately — it also covers producers other than the Claude Code skill, which will keep appearing:

1. Anchor the fallback. Every f"L{...}" site in extract.py emits a single integer, never a range or a list, so the anchor costs nothing for real AST output and stops the fallback from claiming citation spans:

_AST_LOC_RE = re.compile(r"^L\d+$")

2. Use the strict _origin check for fresh chunks in build_merge and merge_raw_extraction, mirroring what the dedup collision guard already does at build.py:992 with a comment describing this exact hazard:

tier_sources = new_ast_sources if n.get("_origin") == "ast" else new_sem_sources

extract() always stamps its own output, so an unstamped item in a fresh chunk is semantic by construction. Only stored items need the shape fallback, and _load_existing_graph has already backfilled _origin on those — so the _is_ast_tier calls over existing_* should stay as they are.

Verification

A standalone regression test over build_merge (semantic re-extract with source_location in None / L460 / L460-L565 / L13,L26-L32, plus an AST-only re-extract that must preserve the semantic layer):

8 assertions fail on unpatched 0.9.46 and on unpatched 0.9.50
all 21 pass with the two changes above

End-to-end on the real graph: re-merging the run's actual semantic chunk into the pre-fix graph.json drops docs/despliegue.md from 29 nodes to 9 and removes the stale claim, matching a hand-audited purge exactly (5,654 nodes both ways).

Happy to open a PR with the two changes and the test if that is useful.

@1sraFigueroa

Still present in 0.9.55, and here is a minimal repro that isolates the variable with a control document in the same run.

build.py:41 is unchanged: _AST_LOC_RE = re.compile(r"^L\d") (still unanchored, so L6220-L6260 matches too), and watch.py:1994 still calls to_json(..., force=True).

Minimal repro

Three files, plus a hand-written graph.json whose doc nodes carry no _origin — the shape any graph built through the Claude Code skill has. The two documents differ only in the shape of source_location:

code.py             def alpha(): return 1
doc_lineloc.md      6 concept nodes, source_location "L10".."L15"
doc_sectionloc.md   6 concept nodes, source_location "§0".."§5"

def sem(i, doc, loc):
    return {'id': f'{doc.replace(".","_")}_c{i}', 'label': f'{doc} concept {i}',
            'file_type': 'concept', 'source_file': doc, 'source_location': loc,
            'community': 0, 'community_name': 'Docs', 'norm_label': f'{doc} concept {i}'}

nodes  = [sem(i, 'doc_lineloc.md',    f'L{10+i}') for i in range(6)]
nodes += [sem(i, 'doc_sectionloc.md', f'§{i}')    for i in range(6)]
nodes += [{'id':'code_alpha','label':'alpha()','file_type':'code','source_file':'code.py',
           'source_location':'L1','community':1,'community_name':'Code',
           'norm_label':'alpha()','_origin':'ast'}]

Then append a function to code.py and run graphify update .:

doc_lineloc.md:    3 nodes total, 0/6 original concept nodes survived
doc_sectionloc.md: 6 nodes total, 6/6 original concept nodes survived

The line-located document's semantic layer is gone, replaced by 3 AST heading nodes. The section-located one is untouched. Same run, same absence of _origin, same file type — only the source_location shape differs.

Real corpus, same run

Documentation repo, ~2.2k nodes, graphify update . after editing two unrelated .md files:

source_file	`source_location` shape	before	after
`plans/2026-07-19-...md`	`L15`, `L6220-L6260`	55 semantic	0 semantic, 60 AST headings (`_origin: ast`, `file_type: document`)
`status/2026-07-20-...md`	`3`, `4.8`	24 semantic	same 24, byte-identical

Nothing surfaced the loss at the time. It came to light only on the next incremental build_merge, which reported edges whose endpoints pointed into that file and no longer resolved.

Two triage notes:

The total went UP, 2230 → 2243, because AST heading nodes outnumbered the semantic nodes they replaced. So even without the force=True on this path, the Prevent partial-chunk overwrite in --update: pre-build node-count assertion + build_merge() helper #479 guard would have passed — this is the exact masking #2229 describes, with concrete numbers.
The graph-health diagnostic stays clean through the loss: 0 dangling, 0 missing, 0 collapsed. Consistent with what @1sraFigueroa reported on the build_merge side.

On the fix

The ^L\d+$ anchor proposed above would not have saved this corpus on its own: L15 is a bare integer and still matches. The _origin stamp (this issue's primary fix) and the strict-_origin check for fresh chunks are what actually cover it. The anchor is still worth having for citation spans like L6220-L6260.

Worth noting for reproduction: the skill's own extraction spec shows line references as valid source_location values, and the generated project CLAUDE.md instructs agents to run graphify update . after every edit — so on a doc-heavy corpus this fires on ordinary use, not on an exotic configuration.

Still present in 0.9.71 (PyPI graphifyy, Linux, Python 3.14), reached through the OMP skill's Part B/C with subagents and no API key.

Corpus: 21 Markdown/YAML documents and 2 images, producing 294 semantic nodes. Every subagent wrote L<line> values in source_location.
The first graphify update (run from a pre-commit hook) deleted 134 of the 294 semantic nodes. They were all concept or rationale nodes carrying an L<line> location. The same run added 119 markdown-AST heading nodes for those documents.
Setting _origin: "semantic" on every unstamped node and edge in the built graph.json (the setdefault approach in fix(skill): stamp _origin on the semantic tier so update cannot delete it (#2843) #3110) and rerunning the same graphify update lost 0 nodes. A second update reported the graph unchanged.
Hyperedges needed no stamp: _rebuild_code keeps a hyperedge when all its members survive, so they came back once their nodes did.

PR Graphify-Labs#3110: fix(skill): stamp _origin on the semantic tier…