Skip to content

opencode2 service startup fails while V1 migration scans a large SQLite database #41739

Open

Description

@tsah

Summary

opencode2 cannot start its background service when migrating a large existing SQLite database. The CLI reports that the server exited after roughly 12 seconds, while the detached server process remains alive in disk wait and scans the database for about an hour. It eventually exits, the service remains stopped, and retrying reproduces the problem.

I searched existing issues and found #41346, but this appears distinct: there is no SQLite syntax error here; startup stalls on a very large database and the client/server lifecycle leaves the service unavailable.

Environment

  • opencode2 v0.0.0-next-17155
  • Node.js v26.4.0
  • Linux 6.1.175-219.359.amzn2023.aarch64 (aarch64)
  • XFS filesystem
  • Existing DB: ~/.local/share/opencode/opencode.db
  • DB size: 29,889,863,680 bytes (~28 GiB), fully allocated
  • SQLite page size/count: 4096 / 7,297,330
  • Journal mode: WAL

Reproduction

  1. Have an existing large ~/.local/share/opencode/opencode.db (about 28 GiB in this case).
  2. Run opencode2.
  3. Observe:
Starting background server...
ERROR (#1): Error: Server process exited with code 1
    at map (native:1:11)
    at <anonymous> (../client/src/effect/service.ts:112:30)
    at service.ensure (src/services/server-connection.ts:66:53)
  1. opencode2 service status reports stopped.
  2. Despite the CLI error, opencode2.exe serve --service remains running. It stays mostly in Linux D state (folio_wait_bit_common) and continues reading the DB. During one observation, read_bytes increased from 67 MB to 294 MB, confirming ongoing I/O.
  3. After about an hour, that process exits. The service is still stopped, and another opencode2 attempt fails the same way.

A concurrent standalone attempt also failed because the first process held the DB lock:

V1 migration failed
EffectDrizzleQueryError: Failed query: INSERT OR IGNORE INTO project ...
SQLiteError: database is locked

That lock error is secondary; it occurred only because the original detached migration process was still running.

Expected behavior

The migration should complete and the service should become available. For large databases, startup should either:

  • wait without incorrectly reporting that the server exited,
  • expose migration progress/status, or
  • fail with an actionable diagnostic.

It should not leave a detached migration process running for an extended period while service status reports stopped.

Actual behavior

The UI cannot start, the CLI says the server exited, a detached process continues scanning the database, and the service never reaches a usable state. Retrying can create lock contention with the still-running migration.

Additional notes

The existing data has been preserved for further diagnostics. I can run targeted, non-destructive SQLite queries if table-level information would help identify why the DB is this large or where migration time is spent.

Activity

  1. opencode-agent commented on Aug 11, 2026

    @opencode-agent
    Contributor

    Could you use sqlite3 against the preserved database and tell us how many rows are in the event table? A read-only command is:

    sqlite3 -readonly ~/.local/share/opencode/opencode.db "SELECT COUNT(*) FROM event;"

    Given the database size, this query may also take a while.

  2. tsah commented on Aug 11, 2026

    @tsah
    Author

    (agent) I ran the read-only query against the preserved database backup:

    sqlite3 -readonly ~/.local/share/opencode/db-backup-20260811T125425Z/opencode.db "SELECT COUNT(*) FROM event;"

    Result: 880,544 rows.

    Timing: 7m33.677s wall time (0m0.020s user, 0m0.224s sys). The original DB remains preserved and unchanged.

  3. davispuh commented on Oct 8, 2026

    @davispuh

    I have exactly same issue. I was using OpenCode V1 and everything was working fine, but now I updated to V2 and:

    $ opencode -v
    opencode v2.0.25
    
    $ opencode
    Starting background server...
    UnknownError: An error occurred in Effect.tryPromise
        at <anonymous> (/$bunfs/root/chunk-y4vc5n58.js:24:5988)
        at ~effect/Utils/internal (/$bunfs/root/chunk-y4vc5n58.js:8:1710)
        at a (/$bunfs/root/chunk-y4vc5n58.js:24:6086)
        at processTicksAndRejections (native:7:39)
        at Tui.run (/$bunfs/root/chunk-ex997hk1.js:98:643)
        at Tui.run (definition) (/$bunfs/root/chunk-ex997hk1.js:93:17212) {
      [cause]: ClientError: Transport: The operation timed out.
          at <anonymous> (/$bunfs/root/chunk-ttfy3wxh.js:2:1017)
          at async <anonymous> (/$bunfs/root/chunk-ttfy3wxh.js:2:1265)
          at processTicksAndRejections (native:7:39) {
        [cause]: TimeoutError: The operation timed out.
      }
    }
    
    $ opencode models
    ClientError: Transport: The operation timed out.
        at <anonymous> (/$bunfs/root/chunk-ttfy3wxh.js:2:1017)
        at unknown
        at processTicksAndRejections (native:7:39)
        at cli.models (/$bunfs/root/chunk-ar5jx5r6.js:7:19388)
        at cli.models (definition) (/$bunfs/root/chunk-0vth0rgw.js:2:961) {
      [cause]: TimeoutError: The operation timed out.
    }
    
    $ opencode serve --print-logs
    timestamp=2026-10-08T15:50:17.451Z level=ERROR run=542bc1c0 message="V1 migration failed" cause="Cause([Die(EffectDrizzleQueryError: Failed query: \n          INSERT OR IGNORE INTO project (id, worktree, time_created, time_updated, time_active, sandboxes)\n          VALUES (?, ?, ?, ?, ?, '[]')\n        \nparams: global,/,1791474612442,1791474612442,1791474612442 (cause: Cause([Fail(effect/sql/SqlError: Failed to execute statement (cause: effect/sql/SqlError/LockTimeoutError: Failed to execute statement (cause: SQLiteError: database is locked)))])))])" role=server
    server listening on http://127.0.0.1:4096
    server password  ...
    
    $ opencode service status
    stopped
    
    $ opencode api get /api/info
    Error: Timed out waiting for the background service to start
        at <anonymous> (/$bunfs/root/chunk-gtfaztdf.js:3:6979)
        at service.ensure (/$bunfs/root/chunk-12bpx8hy.js:3:732)
        at service.ensure (definition) (/$bunfs/root/chunk-gtfaztdf.js:3:5523)
        at cli.server-connection.resolve (/$bunfs/root/chunk-a6y7s241.js:2:1066)
        at cli.server-connection.resolve (definition) (/$bunfs/root/chunk-12bpx8hy.js:2:402)
        at cli.api (/$bunfs/root/chunk-ar5jx5r6.js:7:19388)
        at cli.api (definition) (/$bunfs/root/chunk-a6y7s241.js:2:1026)
    timestamp=2026-10-08T16:29:10.996Z level=INFO run=5cf4c67a message="cli starting" version=2.0.25 channel=latest local=false args="[\"serve\",\"--service\"]" role=cli
    timestamp=2026-10-08T16:29:11.298Z level=INFO run=5cf4c67a message="Sent HTTP response" http.span=4 role=server http.method=GET http.url=/api/info http.status=503
    timestamp=2026-10-08T16:29:16.374Z level=ERROR run=5cf4c67a message="V1 migration failed" cause="Cause([Die(EffectDrizzleQueryError: Failed query: \n          INSERT OR IGNORE INTO project (id, worktree, time_created, time_updated, time_active, sandboxes)\n          VALUES (?, ?, ?, ?, ?, '[]')\n        \nparams: global,/,1791476951367,1791476951367,1791476951367 (cause: Cause([Fail(effect/sql/SqlError: Failed to execute statement (cause: effect/sql/SqlError/LockTimeoutError: Failed to execute statement (cause: SQLiteError: database is locked)))])))])" role=server
    timestamp=2026-10-08T16:29:27.990Z level=WARN run=137d7fb1 message="Background service is unresponsive; recovery cannot preserve persistent terminals" role=cli
    timestamp=2026-10-08T16:29:21.673Z level=INFO run=852a0546 message="cli starting" version=2.0.25 channel=latest local=false args="[\"serve\",\"--service\"]" role=cli
    timestamp=2026-10-08T16:29:27.028Z level=ERROR run=852a0546 message="V1 migration failed" cause="Cause([Die(EffectDrizzleQueryError: Failed query: \n          INSERT OR IGNORE INTO project (id, worktree, time_created, time_updated, time_active, sandboxes)\n          VALUES (?, ?, ?, ?, ?, '[]')\n        \nparams: global,/,1791476962018,1791476962018,1791476962018 (cause: Cause([Fail(effect/sql/SqlError: Failed to execute statement (cause: effect/sql/SqlError/LockTimeoutError: Failed to execute statement (cause: SQLiteError: database is locked)))])))])" role=server
    timestamp=2026-10-08T16:29:38.636Z level=WARN run=137d7fb1 message="Background service is unresponsive; recovery cannot preserve persistent terminals" role=cli
    timestamp=2026-10-08T16:29:32.281Z level=INFO run=97e53543 message="cli starting" version=2.0.25 channel=latest local=false args="[\"serve\",\"--service\"]" role=cli
    timestamp=2026-10-08T16:29:37.656Z level=ERROR run=97e53543 message="V1 migration failed" cause="Cause([Die(EffectDrizzleQueryError: Failed query: \n          INSERT OR IGNORE INTO project (id, worktree, time_created, time_updated, time_active, sandboxes)\n          VALUES (?, ?, ?, ?, ?, '[]')\n        \nparams: global,/,1791476972647,1791476972647,1791476972647 (cause: Cause([Fail(effect/sql/SqlError: Failed to execute statement (cause: effect/sql/SqlError/LockTimeoutError: Failed to execute statement (cause: SQLiteError: database is locked)))])))])" role=server
    timestamp=2026-10-08T16:29:49.247Z level=WARN run=137d7fb1 message="Background service is unresponsive; recovery cannot preserve persistent terminals" role=cli
    timestamp=2026-10-08T16:29:42.918Z level=INFO run=cf9c5a74 message="cli starting" version=2.0.25 channel=latest local=false args="[\"serve\",\"--service\"]" role=cli
    timestamp=2026-10-08T16:29:48.266Z level=ERROR run=cf9c5a74 message="V1 migration failed" cause="Cause([Die(EffectDrizzleQueryError: Failed query: \n          INSERT OR IGNORE INTO project (id, worktree, time_created, time_updated, time_active, sandboxes)\n          VALUES (?, ?, ?, ?, ?, '[]')\n        \nparams: global,/,1791476983257,1791476983257,1791476983257 (cause: Cause([Fail(effect/sql/SqlError: Failed to execute statement (cause: effect/sql/SqlError/LockTimeoutError: Failed to execute statement (cause: SQLiteError: database is locked)))])))])" role=server
    
    timestamp=2026-10-08T16:42:51.334Z level=INFO run=3bb68aeb cause="InterruptError: All fibers interrupted without error\n    at ServerProcess.start (/$bunfs/root/chunk-w0rgwtvq.js:2:7408)\n    at ServerProcess.start (definition) (/$bunfs/root/chunk-pywbvjpj.js:447:25755) {\n  [cause]: InterruptCause: The fiber was interrupted by:\n      at fiber (#419)\n}" http.span=105 role=server http.method=GET http.url=/ http.status=200
    timestamp=2026-10-08T16:42:51.337Z level=INFO run=3bb68aeb cause="InterruptError: All fibers interrupted without error\n    at ServerProcess.start (/$bunfs/root/chunk-w0rgwtvq.js:2:7408)\n    at ServerProcess.start (definition) (/$bunfs/root/chunk-pywbvjpj.js:447:25755) {\n  [cause]: InterruptCause: The fiber was interrupted by:\n      at fiber (#420)\n}" http.span=151 role=server http.method=GET http.url=/ http.status=200
    
    

    I don't know what's correct/best way to recover from this but I tried bunch of things and eventually it succeeded and everything started working.

    I think it would be best if there was simple command opencode migrate that could be ran alone without anything else and let it finish however long it takes (could be hours).

  4. joe-ieta commented on Oct 19, 2026

    @joe-ieta
    Contributor

    Description

    Confirmed migration error on Windows x64, not a feature request. Reopening Desktop kept reaching a failed background service:

    2026-10-18T07:27:54.377Z background service boot failed
    Cause([Die(SQLiteError: disk I/O error)])
    GET /api/info -> 500
    GET /api/event -> 503
    

    The user reported 2.0.23 after upgrading from 1.x; the exported diagnostic bundle identifies 2.0.32. They returned to 1.19.3.

    Read-only snapshot after the incident: database 14,942,654,464 bytes (~13.9 GiB), 3,648,109 pages, 3,322,094 free pages (~91%), and 165 sessions in each of session and session_v2. Migration was already marked completed at inspection; the event-row count at the original failure is unknown.

    Steps to reproduce the confirmed cleanup defect

    1. Use an isolated legacy database with a session/message and 2,500 event rows (~10 MiB payload).
    2. Add a BEFORE DELETE trigger that raises ABORT on event_1001, then start V2.
    3. The real 2.0.26 executable reports migration error; reopening the database still finds 2,500 events. The first 1,000 deletions rolled back because every batch shares one transaction.
    4. With per-batch commits, the same error leaves 1,500 events. Drop the test trigger and restart: migration completes, the session is readable through HTTP, and another restart preserves its history.

    The regression test is here. Run from packages/core:

    bun run test test/v1-migration.test.ts --test-name-pattern "commits event cleanup"

    This establishes the transaction/recovery defect, not the underlying filesystem cause of the original I/O error. #41781 is closed and unmerged; a focused fix will keep existing cleanup semantics and only commit its batches separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions