Skip to content

Proposal: auto-downgrade or default-off callgraph_store for non-git/large dirs to avoid GB-scale callgraph indexing and high I/O #268

Closed

Description

@ICEY16360

Description

Summary

callgraph_store (callgraph persistence) defaults to true in 0.52.2, and unlike search_index/semantic_search it has no automatic downgrade for $HOME / non-git large directories. This still triggers a full cold-build on <home-dir> or <large non-git directory>, producing GB-scale disk usage and sustained high I/O. We propose either extending callgraph_store with a symmetric downgrade, or changing its default to false while allowing per-project opt-in.

This issue is filed as a feature proposal; the rationale: from the available docs and source we cannot determine whether callgraph_store defaulting on (and lacking downgrade for non-git/home roots) is an omission, so we do not presume it a defect—only a behavior-change suggestion, easier for maintainers to triage.

Environment

  • @cortexkit/aft-opencode 0.52.2 (aft binary v0.52.2)
  • Linux (VM), editor runs a persistent opencode serve
  • Config ~/.config/cortexkit/aft.jsonc is empty by default (search_index: false, semantic_search: false); callgraph_store stays at default true

Symptoms

After a <long-running editor host> launched a persistent serve with <home-dir> (or <large non-git directory>) as project_root, the following was observed:

  • Disk filled by the callgraph store (measured up to GB-scale, single project key)
  • Sustained high I/O (measured up to tens-of-thousands of IOPS, block device util 60%+, single core CPU pegged), machine temperature rose significantly
  • /aft-status shows search_index/semantic_search already auto-disabled due to degraded_reasons: ["home_root"], but callgraph_store still enabled

Clearing the index cache under ~/.local/share/cortexkit/aft brought it back down, but as long as project_root remains <home-dir>/<large non-git directory> with callgraph_store:true, a restart reproduces the cold-build.

Root Cause

  1. search_index/semantic_search default false and have a home_root auto-fuse (handle_configure sets degraded), so already mitigated.
  2. callgraph_store defaults true (aft.schema.json: default true), with no equivalent home_root/non-git fuse; for non-git roots it is only configure warm deferred (deferred, not disabled), eventually still bounded cold_build.
  3. Users can hardly perceive that callgraph_store is the only heavy index enabled by default.
  4. When a serve starts with <home-dir> (or any non-git directory) as project_root, AFT takes that directory as the whole callgraph build scope; the non-git root lacks .gitignore constraints (inferred: without git_common_dir, bounded cold_build enumerates files from project_root), and observations show node_modules/.cache etc. get included — i.e. home/non-project directories being treated as index roots and fully indexed is the direct trigger of the runaway growth.

Reproduction

  1. Empty config (don't set callgraph_store), start opencode serve with <home-dir> or any non-git directory containing lots of node_modules/.cache as project_root.
  2. Trigger one aft call (e.g. read), watch aft log callgraph store ... cold_build and /aft-status.
  3. du -sh ~/.local/share/cortexkit/aft keeps growing to GB-scale.

Impact

  • On by default, violating the principle of least surprise; small projects unaffected, but large directories / mis-started cwd directly saturate disk and I/O.
  • The real cost of turning callgraph_store off is only: aft_callgraph unavailable, aft_zoom({callgraph:true}) no annotations, aft_inspect's dead_code reports callgraph_unavailable; cycles/unused_exports/duplicates/todos, ast_grep_search/replace, LSP goto/references, outline/zoom are all unaffected — so the benefit/cost of enabling by default is not worth it.

Suggestions

The following two approaches both achieve "off by default, opt-in on demand"; please pick one:

Option A (symmetric downgrade, zero breaking): Extend the home_root fuse that callgraph_store already shares with search_index/semantic_search. We suggest adding in handle_configure a new heuristic: when project_root is a non-git root and file scale is large (specific threshold, or heuristic hit like node_modules/.cache, to be decided by implementation), automatically set callgraph_store: disabled and record degraded_reasons. This downgrade may still be explicitly overridden per project:

// <project>/.cortexkit/aft.jsonc
{ "callgraph_store": true }

or finer-grained index.roots: [{ "path": "<project>", "indexes": ["callgraph"] }] to enable on demand.

Option B (change default, breaking): Change the global default of callgraph_store from true to false, guiding per-project opt-in:

// <project>/.cortexkit/aft.jsonc
{ "callgraph_store": true }

also achieves "off by default, opt-in on demand"; the cost is that after upgrade all projects lose callgraph by default and must enable per project, but it buys complete mitigation for mis-start scenarios.

Both approaches preserve the fine-grained control of callgraph_chunk_size / index.roots.

Use case

No response

Activity

  1. aft-alfonso commented on Aug 27, 2026

    @aft-alfonso

    Verified at source and accepted in two halves:

    Home roots — you found a real asymmetry. handle_configure fuses search_index and semantic_search on a home root (degraded_reasons: ["home_root"]) but callgraph_store has no fuse arm — home-root warming is only deferred, so a persistent serve rooted at $HOME eventually cold-builds the store anyway. The symmetric fuse is being implemented now: callgraph_store force-disables on home roots with the same disclosure, scheduling gated at the chokepoint so no cold build is ever admitted, and callgraph-dependent surfaces (inspect dead-code, navigation) degrade through the existing callgraph_store: false paths with an honest disabled status instead of a retry loop.

    Non-git / large directories — staying buildable by design. v0.52.2 shipped bounded resumable callgraph builds (streaming enumeration, ~tens-of-MB working set instead of the earlier GB-scale/OOM behavior) plus a crash breaker that suspends repeatedly-failing builds, precisely so non-git directories can be indexed safely — supporting document-directory use cases beyond code. So the default stays true there, with callgraph_store: false available per-project for opt-out. If you're seeing GB-scale stores on a non-home non-git root on 0.52.2+, that would be its own bug — a fresh report with the store path and size would be very welcome.

    The home fuse rides the next patch release.

  2. aft-alfonso commented on Aug 27, 2026

    @aft-alfonso

    The home-root fuse is on main and CI-verified — rides the next patch release:

    • callgraph_store now auto-disables on home roots exactly like search_index and semantic_search, with the same degraded_reasons: ["home_root"] disclosure and a log line naming the remedy (open a project subdirectory).
    • The fuse gates scheduling, not just the flag: no cold-build admission ever occurs on a fused root, and callgraph-dependent surfaces (inspect dead-code, navigation commands, warmup) report a terminal "disabled for home root" state instead of a retry loop.
    • Non-git, non-home directories keep the v0.52.2 bounded-build + crash-breaker behavior by design, per the earlier note.

    Thanks for the precise report — the asymmetry was real and your root-cause section was accurate down to the config lines.

  3. aft-alfonso commented on Aug 27, 2026

    @aft-alfonso

    Released in v0.53.0 — callgraph_store now auto-disables on home-directory roots exactly like the search and semantic indexes (one symmetric fuse), so opening $HOME in an editor no longer triggers GB-scale callgraph builds. Callgraph-dependent surfaces report a terminal disabled state for the root instead of retrying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions