Skip to content

[FEATURE]: Per-session-tree database sharding to eliminate SQLite lock contention #20935

Closed as not planned

Description

@sjawhar

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Note: #14212 proposes replacing SQLite with Postgres. This proposal is complementary — it shards the existing SQLite storage to eliminate lock contention without changing the storage engine.

Describe the enhancement you want to request

Split message/part/todo storage into separate SQLite databases grouped by session tree (root session + all its subagent children), keeping a small global DB for session metadata, accounts, and projects.

Why: The single global opencode.db causes SQLITE_BUSY errors when multiple concurrent processes write to it (#19521, #15188, #14970, #4251). WAL mode allows only one writer at a time — with many concurrent sessions (e.g. parent + subagents, or multiple terminals), writers overwhelm the single write lock.

Why session tree as the shard key: Every query on MessageTable and PartTable already filters by session_id — there are zero cross-session joins. Parent/child sessions (from subagent spawning) are grouped together since they're active simultaneously and represent a single unit of work. Contention within a tree (parent + 2-5 subagents) is well within SQLite's comfort zone.

Layout:

~/.local/share/opencode/
├── opencode.db              # Global: session list, account, project, workspace (small, rarely written)
└── sessions/
    ├── ses_root_001.db      # Messages, parts, todos for this session tree
    └── ses_root_002.db      # Fully isolated from other trees

Implementation sketch:

  1. Add root_session_id to SessionTable — root sessions reference themselves, children inherit from parent
  2. Add Database.session(rootSessionID) — lazily opens per-tree connections with same PRAGMA config
  3. Route message/part/todo writes through session-scoped DB
  4. Migration: new sessions use sharding; old data stays in global DB with fallback reads

Companion PRAGMA improvements (independent, can land first):

  • busy_timeout: 5000 → 30000ms
  • mmap_size: 0 → 2GB for faster reads on large DBs
  • SyncEvent.run(): use "deferred" instead of "immediate" transactions when OPENCODE_EXPERIMENTAL_WORKSPACES is off
  • Periodic WAL checkpoint + TRUNCATE on close

Activity

  1. added
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    perfIndicates a performance issue or need for optimization
    on Apr 3, 2026
  2. github-actions commented on Apr 3, 2026

    @github-actions
    Contributor

    This issue doesn't fully meet our contributing guidelines.

    What needs to be fixed:

    • The title should start with [FEATURE]: for feature requests (currently uses feat:)
    • Missing the required verification checkbox confirming the feature hasn't been suggested before
    • The description appears to be AI-generated (extensive wall of text with tables, architecture diagrams, implementation sketches) — please summarize the request in your own words

    Please edit this issue to address the above within 2 hours, or it will be automatically closed.


    This issue might be a duplicate of existing issues. Please check:

    If you believe this was flagged incorrectly, please let a maintainer know.

  3. changed the title [-]feat: per-session-tree database sharding to eliminate SQLite lock contention[/-] [+][FEATURE]: Per-session-tree database sharding to eliminate SQLite lock contention[/+] on Apr 3, 2026
  4. github-actions commented on Apr 3, 2026

    @github-actions
    Contributor

    Thanks for updating the issue! It now follows the contributing guidelines and looks great. We appreciate the detailed explanation of the problem, the proposed solution, and the references to related issues.

  5. assigned and unassigned on Apr 26, 2026
  6. removed
    perfIndicates a performance issue or need for optimization
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on May 3, 2026
  7. github-actions commented on Jul 3, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions