Skip to content

Determine how to not need multiple codedb instances — or replace codedb effectively #2225

Closed

Description

@dezren39

Problem

codedb (the MCP code-index/search server in the fleet's mcpx config, .mcpx.json → command: /home/user/bin/codedb) runs as one process per client session. Observed 2026-10-29: ~7 simultaneous codedb mcp children on the laptop, parented by the OpenClaw gateway supervisor and MCP-client node processes. There is no codedb systemd unit in either manager — process count scales with session count, and each instance presumably maintains its own index state.

Tonight its mcpx entry was parked (PR #2220: moved out of mcpServers into a top-level disabledMcpServers key, definition retained verbatim) pending this determination. Existing per-session children retire with their sessions; the park only stops new spawns once the PR lands and the primary checkout is pulled.

Determination to make

Evaluate, with evidence:

  • (a) One shared codedb service — a single long-lived codedb instance (daemon/service shape) that all MCP clients connect to, instead of per-session children. What does codedb support today (transport, shared state, concurrency), and what config changes does each client need?
  • (b) Replace codedb with the fleet's built-in embedding machinery — OpenCode core already ships a local embedding stack (llama.cpp + Qwen3-Embedding tiers, packages/core/src/messaging/embedding.ts in developing-today/opencode), but today it indexes messages and session transcripts for messaging search, not source code. Atlas (hatch-personal workspace/turso-atlas/) indexes conversations + repo docs, not source code either. Is extending one of these to code search realistic, and what would it cost vs (a)?
  • (c) Another existing fleet component — e.g. mcpx's own catalog embeddings are tool-catalog-scoped (semantic tool search), not code-scoped; state plainly if nothing existing fits.

Acceptance

Related

Acceptance criteria

  • A determination is recorded on this issue with evidence for each option: (a) one shared codedb service, (b) a replacement for codedb, or (c) per-session instances by design — including measured per-instance cost (memory/index state) from the observed ~7-instance state.
  • The chosen option is either implemented (PR linked, mcpx config updated from the parked state in PR config: park qwen models (cline-pass/commandcode) + park codedb in mcpx — stacked on #2198 #2220) or filed as a linked follow-up issue with the determination as its spec.
  • The interim park state is explicitly resolved: codedb either returns to mcpServers under the chosen model or stays parked with the reason recorded here.

Activity

  1. added
    kind/featureNew feature or capability request
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    effort/mMedium: expected under 1 day of work
    on Oct 29, 2026
  2. developing-today commented on Oct 29, 2026

    @developing-today
    Contributor
  3. added
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Oct 29, 2026
  4. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    Intake review

  5. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 1, 2026
  6. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    Triage verdict — task 3384 (lane C, 2026-11-01): still LIVE; routing the determination investigation.

    Fresh state checks:

    Routing: the determination itself is the next unit of work — it needs evidence this triage does not gather (codedb's transport/concurrency support, measured per-instance memory/index cost from the observed ~7-instance state, the OpenCode embedding stack's realistic extension cost, and the codebase-memory candidate). A follow-up queue task has been enqueued to produce the written determination on this issue — options (a)/(b)/(c) with evidence, the per-client config changes the recommended path implies, and the explicit park resolution (lift or make permanent) — and to route the resulting implementation as its own packet/issue. This triage does not pre-judge the outcome.

  7. 22 remaining items

  8. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — triaged (task 5908, OpenChamber scheduled lane)

  9. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 2, 2026
  10. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  11. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done — determination for #2225 (task 5226, queue worker lane C)

    Determination: one shared codedb daemon per project root + a thin MCP bridge — park lifts in the new shape

    Measured per-instance cost (laptop, 2026-11-02, this run)

    • codedb is a 6.9 MB static binary (~/bin/codedb, Apr 2026 build). Zero codedb processes are running right now — the PR config: park qwen models (cline-pass/commandcode) + park codedb in mcpx — stacked on #2198 #2220 park is effective — and there is no codedb systemd unit in either the system or user manager.
    • On-disk index state is per project, not per instance: ~/.codedb/projects/<hash>/ totals 253 MB across 9 projects (largest 148 MB). Disk does not multiply with sessions.
    • Memory does. Measured with codedb search IdentityStore against ~/git/code (the largest indexed project), peak RSS via ru_maxrss:
      • Cold (re-index): 2,135 MB peak, 22.6 s indexing inside a 24.6 s call.
      • Warm (index current): 1,104 MB peak, ~21 s wall (index load/validate dominates; the search itself is ~1 ms).
    • So each per-session stdio instance costs roughly 1.1 GB RSS warm / 2.1 GB cold-peak for the big repo. The ~7 concurrent instances observed at filing therefore represented on the order of 8–15 GB of aggregate RSS — that multiplication, not disk and not the tool's quality, is the problem to solve.

    Option (a) — shared codedb service: SUPPORTED BY THE BINARY, with one gap

    • codedb --help lists, alongside mcp (JSON-RPC/MCP over stdio), a serve command: "HTTP daemon on :7719". Binary string evidence shows the daemon's routes mirror the query surface: /tree, /outline, /search, /searchq, /word, /hot, /health. One daemon holds the index in memory and any number of HTTP clients share it — concurrency is the daemon's normal shape.
    • The gap: there is no MCP route on the daemon (no /mcp endpoint; mcp is stdio-only). MCP clients (mcpx, OpenCode, OpenClaw) cannot point at :7719 directly. Sharing therefore needs a thin stdio MCP→HTTP bridge: a small stateless shim per session that exposes codedb's MCP tools and forwards to the local daemon. The shim is per-session but negligible (no index, no state) — the expensive part stops multiplying.
    • One daemon is per project root (codedb [root] serve). The fleet's working set is bounded: one daemon per primary clone (code, opencode, openchamber, nix — ≤4 daemons, ~1.1 GB for code, less for the smaller repos), versus an unbounded per-session count today.

    Option (b) — replace with the fleet's embedding machinery: NOT RECOMMENDED

    • OpenCode's embedding stack is real (packages/core/src/messaging/embedding.ts in developing-today/opencode, ~43 KB, llama.cpp + Qwen3-Embedding) but it indexes messages and session transcripts; Atlas indexes conversations + repo docs. Neither indexes source code.
    • codedb's value is largely structural and exact — symbol find, outline, trigram search, inverted-index word — which an embeddings-only replacement does not reproduce; semantic search would be a capability downgrade dressed as a swap.
    • Extending the messaging embedding stack to a code indexer (chunking, symbol extraction, per-repo index lifecycle, MCP surface) is a new subsystem, larger than the bridge in (a) by an order of magnitude, to end at a weaker tool. Rejected on cost and fit.

    Option (c) — another existing component: NONE SOLVES THE ACTUAL PROBLEM

    • codebase-memory is already in .mcpx.json mcpServers (npx -y codebase-memory-mcp) — but it spawns per session via npx, exactly the multiplication shape this issue exists to kill, and its parity with codedb's symbol/trigram surface is unevaluated. It may be a fine complement; it is not an answer to instance multiplication.
    • mcpx catalog embeddings are tool-catalog-scoped (semantic tool search), not code-scoped — stated plainly: not a candidate.
    • fff servers in the config are file-finding, not symbol/code indexing.

    Recommended path and the changes it implies

    1. Daemon: a user systemd unit running codedb serve per primary project root (start with ~/git/code; add roots as needed). Unit lives with the fleet's other user services; config change lands via the normal repo/PR route for units.
    2. Bridge: a small stdio MCP shim (fleet-owned, in developing-today/code) that forwards codedb tool calls to http://127.0.0.1:7719 for the session's project root. First verification step of the follow-up: confirm serve's exact request/response shapes against the MCP tool surface (names/args), and confirm daemon behaviour when the index is stale (auto-reindex observed in CLI runs).
    3. mcpx config: codedb moves from disabledMcpServers back to mcpServers with command = the bridge (env CODEDB_NO_TELEMETRY=1 retained). OpenClaw / OpenCode clients need no codedb-specific change beyond consuming mcpx config as they already do — the bridge is the mcpx entry.

    Interim park resolution

    The PR #2220 park is temporary, not permanent: it stays in force until the follow-up lands (codedb is safer parked than multiplying), and it is lifted in the new shape — daemon + bridge — by that follow-up's config PR. It does not become permanent, because the determination found a supported shared shape; codedb is not being replaced.

    Routed follow-up

    Implementation is filed as a linked issue on this repo with this determination as its spec, and enqueued as a queue packet; both are named in the completion note below / the follow-up comment on this issue. #2225 itself stays open until the follow-up lands and the park is actually lifted.

  12. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  13. added a commit that references this issue on Nov 2, 2026
  14. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done — determination implemented and merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

effort/mMedium: expected under 1 day of workkind/featureNew feature or capability requestmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/normalOrdinary priority (the default)

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions