Repository navigation
Invalid-confidence warning is permanent: _fold_edge_aliases normalizes numeric legacy confidences but not string ones (high/medium/confirmed) #4495
Description
Summary
Any graph whose semantic (LLM) pass emitted a string confidence outside the enum re-warns forever on every command:
[graphify] Extraction warning (259 issues): 259x other schema issue
(e.g. Edge 7335 has invalid confidence 'confirmed' - must be one of ['AMBIGUOUS', 'EXTRACTED', 'INFERRED'])
_fold_edge_aliases() in graphify/build.py normalizes only numeric legacy confidences; free-form strings pass through untouched and are never repaired by any CLI command.
Environment
- graphify / graphifyy 0.9.91, Windows, Python 3.15
- Corpus: mixed code + docs (OpenSpec markdown), 2777 nodes / 6648 edges, 153 communities
- The graph was built by an earlier full build (semantic pass = host agent);
graphify update .only re-extracts code, so the offending edges persist.
Impact
- 259 edges carry a non-enum
confidence:high= 154,confirmed= 93,medium= 6,inferred= 3,medium-high= 2,medium-low= 1. - All 259 have
_origin: semantic, all haveconfidence_score, and all come from documents (253.md, 6.yaml). graph["hyperedges"]carries the same class of bad values: floats (e.g.0.75,0.85,0.9) and strings (high,medium,inferred,medium-high).- The warning is re-emitted on every
graphify update,query,explain,label, … and is not fixable from the CLI.
Root cause
graphify/build.py:
def _fold_edge_aliases(edge: dict) -> None:
...
_conf = edge.get("confidence")
if isinstance(_conf, (int, float)) and not isinstance(_conf, bool):
if edge.get("confidence_score") is None:
edge["confidence_score"] = float(_conf)
edge["confidence"] = "INFERRED"
if not edge.get("confidence") and edge.get("confidence_score") is not None:
edge["confidence"] = "INFERRED"The docstring already reasons: "numeric confidences only ever came from the LLM semantic pass, and LLM-derived edges are INFERRED by definition" and "Without this fold, every reload of a pre-enum graph re-warns once per legacy edge, forever." That reasoning applies verbatim to string legacy values (high / medium / medium-high / medium-low / confirmed / lowercase inferred), but they are not folded — so they re-warn forever, exactly as the docstring warns for numerics.
The extraction prompt (references/extraction-spec.md) asks for "confidence":"EXTRACTED|INFERRED|AMBIGUOUS" plus a separate confidence_score float, so this string vocabulary is model drift, not a documented format.
Suggested fix
Extend the fold to non-enum strings, case-insensitively, mapping to INFERRED (LLM-derived by definition), moving a parseable value into confidence_score:
_ALLOWED = {"AMBIGUOUS", "EXTRACTED", "INFERRED"}
_conf = edge.get("confidence")
if isinstance(_conf, (int, float)) and not isinstance(_conf, bool):
if edge.get("confidence_score") is None:
edge["confidence_score"] = float(_conf)
edge["confidence"] = "INFERRED"
elif isinstance(_conf, str) and _conf.upper() not in _ALLOWED:
try:
if edge.get("confidence_score") is None:
edge["confidence_score"] = float(_conf)
except ValueError:
pass
edge["confidence"] = "INFERRED"
if not edge.get("confidence") and edge.get("confidence_score") is not None:
edge["confidence"] = "INFERRED"The same fold should be applied to graph["hyperedges"], which currently is not normalized at all.
Steps to reproduce
- Build a graph whose semantic pass emits a non-enum string confidence (e.g.
{"confidence": "high", "confidence_score": 0.85}). - Run any graphify command (
graphify query "...",graphify update .). - Observe the "invalid confidence" schema warning; re-running never clears it and the edge values stay unchanged.
Workaround
Post-process graph.json: map every non-enum confidence to INFERRED (and mirror the same mapping into graph.hyperedges).
Activity
github-actions commentedon Oct 16, 2026
Thanks for opening this issue, @aglit87. A maintainer will take a look soon.
If you would like to discuss it in real time, come say hi on our Discord server. For longer-form questions and ideas there is also GitHub Discussions.
To help us triage, please make sure the report includes what you expected, what actually happened, and the steps (and a small sample) to reproduce it.
omsenjalia commentedon Oct 18, 2026
This looks fixed on the
v8branch by 5fec9d32 (Add regression tests for the drifted confidence fold). Could a maintainer confirm and close?