Skip to content

watcher.ignore never reaches the embedded fff engine — per-dir inotify watches on every indexed dir (192K watches / 4.5GB RSS on a large root); root .ignore file is the only effective channel #37740

Closed as not planned

Description

@zeroclaw42

Summary

watcher.ignore in opencode.json(c) never reaches the embedded fff engine. fff's scanner honors only gitignore-family channels (.gitignore, .git/info/exclude, .ignore), and on Linux its watcher then registers one non-recursive inotify watch for every directory the picker indexed. On a large multi-repo workspace root this produced 192,033 inotify watches and 2.5–4.5 GB RSS in opencode serve, no matter what watcher.ignore contained.

Placing a ripgrep-style .ignore file at the project root is a complete cure: watches dropped 192,033 → 778 and steady-state RSS ~4.5 GB → 448 MB, with file search still working correctly. This is the root cause behind the class of reports like #23190 (closed stale, never root-caused: "only an actual .gitignore inside a real .git repo avoids it" — that observation was correct, and this is why), and it's the Linux/memory sibling of #32511 (fff content-indexing $HOME on macOS). Likely also relevant to #32930 / #16610 (inotify exhaustion) and the memory megathread #20695.

Environment

opencode 1.17.13 (npm global), server mode: opencode serve --hostname <ip> --port 4096
OS Ubuntu, kernel 6.17.0, x86-64, 32 cores
Project root /data — a multi-repo orchestration workspace, 610,201 directories total
Config watcher.ignore set (see matrix below)

Checked release notes through 1.18.3 — no watcher/picker/memory fix in the range, so this should still reproduce on current.

Evidence chain

1. The watches belong to fff, and watcher.ignore does not affect them.

Resolving every watched inode (/proc/<pid>/fdinfo/<inotify-fd> → inode → path) accounted for 100% of the watches. The top watched subtrees were exactly the directories listed in watcher.ignore:

14036  /data/state/...        ← listed in watcher.ignore
 5893  /data/runner/...       ← listed in watcher.ignore
 ...   (16 more runner/CI trees, all listed)

2. Config-glob matrix (converged watch counts, each state held stable ≥3 min before reading; the scan takes ~5 min to converge after start):

watcher.ignore form converged inotify watches
absolute paths (/data/repos/**, …) 192,033 (2 inotify instances)
relative (**/repos/**, …) 96,060 — halved
relative + bare (repos/** added) 192,238 — worse again
any of the above + .ignore file at root 778 (1 instance)

The halving is the tell: there are two watcher instances — opencode's own (which does honor watcher.ignore, with relative globs) and fff's (which never sees it). Config changes can only ever fix the first one.

3. Mechanism, from fff-core source (fff.nvim's Rust engine, embedded in the opencode binary — the fff-scan / fff-watcher-own / notify-rs threads):

  • crates/fff-core/src/walk/ripgrep.rs — the walk is built with .git_ignore(true) .git_exclude(true) .git_global(true) .ignore(true); custom overrides exist only for non-git roots and only from a hardcoded IGNORED_DIRS const (node_modules, __pycache__, …). There is no channel for embedder-supplied patterns like watcher.ignore.
  • crates/fff-core/src/watcher/background_watcher.rs — on Linux (no kernel recursion in inotify) it iterates picker.for_each_dir(...) and registers RecursiveMode::NonRecursive per directory. Watch set = indexed set. Whatever the scan doesn't prune, the watcher pays for at ~1 KB kernel memory per watch, plus the picker's own index memory.

4. The .ignore cure, verified end-to-end:

.ignore at the project root with anchored top-level patterns:

/repos/
/runner/
/state/
/logs/
...

Result, reproduced exactly across restarts: 778 watches, 650 MB anon / 448 MB RSS (from ~2.5 GB anon), /find/file returns correct results, excluded trees still directly readable via /file/content (only search indexing is affected).

5. fff rescans dynamically — deleting the .ignore file at runtime ballooned watches back to ~192K within minutes, without a restart. (Good news: the reverse means the file can also be added without restarting, though we restarted to reclaim the watches.)

Why this matters beyond memory

In server mode this compounds: on our host the growth ended in the process wedged in mem_cgroup_handle_over_high (uninterruptible sleep, port listening but never accepting — connections queue in the backlog forever) while systemd still reported it healthy. Any deployment that caps the service with MemoryHigh converts this leak into a silent hang rather than a crash-restart.

Suggested fixes (any subset helps)

  1. Wire watcher.ignore (or a dedicated files.ignore) into fff's walk as ignore overrides — the ignore crate's OverrideBuilder already supports this; fff-core would need to accept embedder patterns (currently only the hardcoded IGNORED_DIRS const, and only for non-git roots).
  2. Until then, document that watcher.ignore does not apply to fff and that a root .ignore file is the effective channel. What is the point of watcher.ignore #6740 ("What is the point of watcher.ignore") and watcher.ignore in opencode.json does not prevent hang in non-git dirs with large subtrees #23190 both died on this gap.
  3. Consider a watch budget with graceful degradation (stop adding watches past N, keep serving) — related: fix(core): prevent hang when inotify watches are exhausted #32930, Opencode hangs at startup if a .git repo is present and inotify user instances run out #16610.

Happy to provide the full measurement methodology (inode→path resolution scripts, per-fd watch accounting) if useful.

Activity

  1. github-actions commented on Sep 18, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions