Skip to content

Concurrent opencode run instances deadlock during init (futex, no timeout) — second overlapping instance hangs forever #29395

Closed as not planned

Description

@paat

Summary

When two opencode run processes overlap in time on the same data dir, the instance that starts while another is still alive deadlocks during initialization and never makes progress — no model call, no error, no timeout. It parks on a futex indefinitely until killed externally. A single opencode run, or strictly serial (non-overlapping) runs, always work.

This is not provider/model/auth/network related — see the minimal repro below, which deadlocks with a bogus model (init never even reaches model resolution).

Minimal reproduction

# Two runs that overlap in time. Bogus model on purpose: no valid provider,
# no auth, no network needed. The deadlock is in instance bootstrap, BEFORE
# model resolution — so neither run ever reaches an "unknown model" error.

( timeout 45 opencode run --agent plan -m bogus/nope-1 "hi"; echo "run1 exit=$?" ) &
sleep 3
( timeout 45 opencode run --agent plan -m bogus/nope-2 "hi"; echo "run2 exit=$?" ) &
wait

Observed: both print exit=124 (timed out, killed), empty output. Neither errors on the invalid model.

For contrast:

  • A single opencode run (any real model, full flags) completes in ~7s.
  • Two runs started 5s apart also fail — the second one (which boots while the first is still alive) is the one that hangs. So it's temporal overlap, not a start-time race.

Expected behavior

A second concurrent instance should either run concurrently, or fail fast with a clear "another instance is using this data dir" message — not block forever on a lock with no timeout.

Diagnostics from a hung instance

  • All threads parked in futex_do_wait (one event-loop thread in ep_poll); ~1s of CPU over 7+ minutes, ~0.2% CPU — not computing.
  • Zero TCP connections, no SYN-SENT — it never attempts any network call. (DNS resolves fine.)
  • Its ~/.local/share/opencode/log/*.log stops at loading internal plugin and never reaches the creating instance → llm runtime selected lines that a successful run logs. So it deadlocks in bootstrap/plugin-init.
  • The hung instance holds opencode.db, opencode.db-wal, and opencode.db-shm open — consistent with a lock on the shared SQLite data dir that the second instance waits on indefinitely.

Environment

  • opencode v1.15.10 (latest at time of filing)
  • Linux container, single user, default XDG_DATA_HOME (~/.local/share/opencode)
  • Bun runtime (bundled)

Impact / how I hit this

A multi-provider code-review tool launches several opencode run calls in parallel against the same data dir; the overlapping ones deadlock and only get freed by a wrapping timeout. Workaround on the caller side is to serialize the opencode run invocations, or give each its own XDG_DATA_HOME (with auth.json seeded into it). But the no-timeout deadlock itself looks like an opencode bug worth fixing.

Happy to provide wchan/fd dumps or a fuller debug log on request.

Activity

  1. paat commented on May 26, 2026

    @paat
    Author

    Also reproduces with a single instance — no concurrency — when cwd is a git repo (v1.15.10)

    The same init deadlock (same signature: frozen at loading internal plugin, futex-parked, holds opencode.db/-wal/-shm, never reaches model resolution) also hits a single opencode run with no second instance — triggered purely by cwd being inside a git repository. So "a single opencode run … always works" isn't quite right: single runs work from a non-repo dir but hang from inside a repo.

    Minimal repro (bogus model → no auth/network/valid model; the hang is before model resolution):

    # Hangs (timeout 124), 0 output, log frozen at "loading internal plugin":
    cd "$(mktemp -d)" && git init -q && git commit --allow-empty -qm x
    timeout 30 opencode run --agent plan -m bogus/nope "hi"
    
    # Control — same command from a NON-repo dir fails fast in ~1s:
    cd /tmp && timeout 30 opencode run --agent plan -m bogus/nope "hi"   # => "Model not found: bogus/nope"

    Extra findings:

    • Reproduced across multiple repos (fresh 1-file repo, large real project). Non-repo dirs (/tmp, $HOME) always work in ~1–2s.
    • Hung process has no child git/bun process → in-process deadlock, not a stuck git subprocess.
    • Still hangs with an empty XDG_CONFIG_HOME (no user plugins/config) → vanilla opencode.
    • ~/.local/state/opencode/locks/ is empty during the hang; a fresh XDG_DATA_HOME doesn't help (the trigger is cwd, not the data dir).

    Hypothesis: not concurrency-specific but a re-entrant lock on the shared SQLite data dir during bootstrap — the git/snapshot subsystem (which only initializes inside a repo) appears to open the data dir a second time while the first open's lock is held, self-deadlocking a single process — the same lock two processes contend on in the concurrency repro. wchan/fd dumps available on request.

    Env: opencode v1.15.10, Linux, Bun runtime, default XDG_DATA_HOME.

  2. HrushiYadav commented on Jun 8, 2026

    @HrushiYadav

    Looking into this. The symptoms point to a re-entrant SQLite lock during bootstrap when the snapshot/git subsystem initializes inside a repo. Will investigate and share findings.

  3. HrushiYadav commented on Jun 9, 2026

    @HrushiYadav

    After deep investigation, here's what I found:

    The direct cause: The Semaphore(1) in sqlite.bun.ts that serializes SQLite access has no timeout. Once contention triggers a wait, the fiber blocks forever -- no error, no log, just a futex park.

    Where the semaphore lives: packages/core/src/database/sqlite.bun.ts lines 121-130. Two acquirers share it:

    • acquirer (regular queries): semaphore.withPermits(1)(...) -- brief hold
    • transactionAcquirer (transactions): semaphore.take(1) + scope-scoped release -- held for entire tx lifetime

    Why git repos trigger it: The boot path for git repos is heavier than non-git. project.fromDirectory runs migrateProjectId and saveProjectDirectory as BEGIN IMMEDIATE transactions, plus git subprocess calls (rev-parse, remote, rev-list). After that, bootstrap.run calls plugin.init() which dynamically imports server/server.ts (a heavy import tree). The exact deadlock path likely involves a fiber scheduling interaction between the transaction scope's finalizer and a concurrent semaphore acquire during the bootstrap phase -- the Effect semaphore is not reentrant, so any re-entrant acquire from the same or a dependent fiber deadlocks silently.

    For concurrent instances: Second process opens the same opencode.db with busy_timeout=5000 at the SQLite file level, but the JS semaphore blocks indefinitely at the application level -- the 5s SQLite timeout never gets a chance to fire because the semaphore wait happens before the SQL statement is even issued.

    Fix: PR #31461 adds a 30-second Effect.timeoutFail to both acquirers. This turns the silent hang into a SqlError with a clear message ("Timed out waiting for database transaction lock after 30s -- possible deadlock"). 1 file, 31 lines added.

  4. github-actions commented on Aug 9, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions