[Bug]: Thread snapshot HTTP handlers have no server-side timeout; cold-open herd times out at 6s and floods relay logs with context-canceled warnings

Before submitting

I searched existing issues and did not find a duplicate.
I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

Run the server (official nightly, orchestration V2) against a moderately large statev2.sqlite with many threads.
Cold-open a client with many threads mounted and a cold snapshot cache (fresh profile), so every mounted thread misses its snapshot cache.
Each mounted thread fires GET /api/orchestration/threads/:threadId/bounded with the client's fixed 6 s budget (DEFAULT_BOUNDED_THREAD_SNAPSHOT_TIMEOUT_MS = 6_000, packages/client-runtime/src/state/boundedThreadSnapshotHttp.ts:19), with no cross-thread concurrency bound (threads.ts only has a per-thread applyLock, packages/client-runtime/src/state/threads.ts:235).
The server serializes all of them on the single-permit SQLite client (semaphore = yield* Semaphore.make(1), packages/shared/src/nodeSqliteClient.ts:262) under BEGIN IMMEDIATE, each holding the lane across a multi-table projection read inside sql.withTransaction (getThreadSnapshotWindow, apps/server/src/orchestration-v2/ProjectionStore.ts:4856).
The snapshots queue, all exceed 6 s, and every client request fails with RemoteEnvironmentAuthTimeoutError → unavailable → socket full-snapshot fallback, while the server-side fibers keep running until the client cancels them. cloudflared logs both sides as Incoming request ended abruptly: context canceled.

Expected behavior

Snapshot HTTP handlers bound their work server-side (timeout with a retryable busy signal the client can back off on instead of hanging until cancel), the client limits in-flight snapshot loads, and downstream-client-goes-away relay lines log at debug rather than warning.

Actual behavior

The three thread-snapshot handlers (threadSnapshot, threadBoundedSnapshot, threadHistoryPage in apps/server/src/orchestration-v2/http.ts) have no server-side timeout/deadline — verified by reading the handlers, the ThreadManagementService → Orchestrator → ProjectionStore delegation chain, and traceLocalHandlerWork (tracing only). A slow read holds its SQLite transaction open until the client gives up and cancels.
The client fans out one 6 s bounded request per mounted cache-miss thread with no semaphore, staggering, or batching, so a shell restore is a thundering herd against the single SQLite lane; mass timeouts then stampede onto the socket full-snapshot path the bounded route exists to avoid.
classifyRelayClientOutput (apps/server/src/cloud/ManagedEndpointRuntime.ts:81) maps the resulting ... ERR Incoming request ended abruptly: context canceled ... lines to warning via the generic ERR rule, with no special case (repo-wide grep for the substring: zero hits). Each line becomes a Relay client reported a transport warning log; nothing actionable — warning on this path only logs (observeConnectorOutput, no restart/recovery). In reporter logs this is on the order of ~135 such warnings/day plus repeated enrichment warns for deleted checkouts on wake.

Impact

Major degradation or frequent failure

Version or commit

Local checkout 0678e4e23d (2026-10-07); re-verified each claim against upstream main tip via the contents API on 2026-10-10 (handler file unchanged — no timeout combinator; classifier unchanged; client still 6 s default with no semaphore).

Environment

Linux server, official nightly builds (no fork). Client: web.

Logs or stack traces

Representative relay lines (from cloudflared output observed in reporter logs):

ERR Incoming request ended abruptly: context canceled ...

Server side, each corresponds to a snapshot handler fiber dying on Interrupt after the client's 6 s cancel; client side, RemoteEnvironmentAuthTimeoutError → unavailable → socket fallback.

Workaround

None on official nightlies — snapshot timeouts/knobs are not exposed via config or environment (no *_TIMEOUT/snapshot_* env knobs found in packages/ or apps/), and the relay log level for the spawned cloudflared process is fixed at --loglevel info (ManagedEndpointRuntime.ts:350).

Suggested fix shape (for triage)

Bound the three snapshot handlers server-side (e.g. Effect.timeoutOption ≈ 5 s, inside the client's 6 s budget) and answer a retryable busy signal (HTTP 429) instead of hanging until cancel.
Bound in-flight client snapshot loads (e.g. a small semaphore) and retry once on busy rather than falling straight back to the socket full snapshot.
Special-case Incoming request ended abruptly: context canceled to debug in classifyRelayClientOutput — it means the downstream client went away, not a tunnel fault.

[Bug]: Server: the thread list read blocks the event loop and every other database call for seconds on large databases #14701 covers the sibling shell-snapshot stall (event-loop blocking on large DBs); this issue is the thread-snapshot timeout/herd/log-noise chain. The two compound: any slow snapshot burns the single lane for everyone behind it.
Already landed upstream (not duplicates): bounded snapshots (feat(orchestrator): introduce new orchestrator #2829), FTL/PNC relay classification (fix(server): surface cloudflared FTL/PNC relay logs as warnings, not debug #5076), tunnel recovery (fix(server): recover a deleted tunnel when Cloudflare says "Tunnel not found" #16648).

Consolidated independent reproduction of the shared-writer part of this issue. Verified source at e536bfd33bc3c351997141d93790df75eeded7b9; the relevant projection/SQLite files are unchanged from the original audit at ec1313c8f8333292506f05a355bb72d308a455d8.

A tiny synthetic WAL database with one thread and one completed tool item is enough: a worker connection takes BEGIN IMMEDIATE, signals that it holds the lock, and releases it after 3,000 ms. The main process then schedules a 1 ms heartbeat and calls the production getThreadSnapshotWindow method. The control wraps that exact method in T3's existing NodeSqliteClient.readOnly helper. No provider, relay, real history or large fixture is needed.

Retained Node 24.21.0, Linux, alternating two rounds:

Arm	Snapshot time	1 ms heartbeat actually fired after
Default, round 1	3,046.7 ms	3,046.8 ms
Reader control, round 1	37.4 ms	1.2 ms
Reader control, round 2	4.6 ms	0.7 ms
Default, round 2	3,041.2 ms	3,041.3 ms

Re-running the same portable synthetic probe during consolidation under Node 26.10.0 gave 3,047.3 ms versus 1.33 ms heartbeat delay. Both returned the same one-item snapshot. The first reader use includes worker startup.

The causal path is:

getThreadSnapshotWindow uses sql.withTransaction without readOnly.
NodeSqliteClient routes unmarked work to the synchronous writer, whose transaction uses BEGIN IMMEDIATE.
Sqlite.ts sets a 5,000 ms busy timeout. The native synchronous wait prevents an Effect timeout or unrelated JS callback from running until it returns.

Minimal experimental sequence for a checkout-level regression test:

seed via T3 migrations + ProjectionStore.apply(thread.created, turn-item.updated)
worker: open same temporary WAL DB; BEGIN IMMEDIATE; signal held
main after held: setTimeout(heartbeat, 1); call getThreadSnapshotWindow
worker: ROLLBACK after 3000 ms
repeat with identical read effect wrapped in NodeSqliteClient.readOnly

The complete checkout probe is included below. The worker hold is an experimental input, not a measured production lock frequency. No multi-second stall appeared in the retained production trace windows, so this is a controlled root-cause reproduction, not a claim that every slow cold-open has this cause. No live database was modified.

Complete checkout-level reproduction (temporary synthetic database only)

Install this checkout's normal development dependencies, save the following as snapshot-contention.mjs, and run from the repository root with Node 24:

node /path/to/snapshot-contention.mjs synthetic results.json 3000 2

The script imports production services from the current working directory. It uses a worker to release the experimental write lock independently of the blocked main loop and removes its synthetic fixture on completion. Do not substitute a live database path for synthetic.

import {createRequire} from 'node:module';
const repo=process.env.T3_AUDIT_REPO??process.cwd();
const req=createRequire(path.join(repo,'apps/server/package.json'));
const Effect=await import(req.resolve('effect/Effect'));
const Layer=await import(req.resolve('effect/Layer'));
const Tracer=await import(req.resolve('effect/Tracer'));
const Deferred=await import(req.resolve('effect/Deferred'));
const Fiber=await import(req.resolve('effect/Fiber'));
const Stream=await import(req.resolve('effect/Stream'));
const SqlClient=await import(req.resolve('effect/sql/SqlClient'));
const ManagedRuntime=await import(req.resolve('effect/ManagedRuntime'));
const DateTime=await import(req.resolve('effect/DateTime'));
const NodeServices=await import(req.resolve('@effect/platform-node/NodeServices'));
const NodeSqlite=await import(path.join(repo,'packages/shared/src/nodeSqliteClient.ts'));
const Store=await import(path.join(repo,'apps/server/src/orchestration-v2/ProjectionStore.ts'));
const Wire=await import(path.join(repo,'apps/server/src/orchestration-v2/WireProjection.ts'));
const History=await import(path.join(repo,'apps/server/src/orchestration-v2/threadHistoryPaging.ts'));
const SqlitePersistence=await import(path.join(repo,'apps/server/src/persistence/Sqlite.ts'));
function openRuntime(filename){
 if(filename.includes('/.t3/userdata/'))throw new Error('Use a copied database only');
 return ManagedRuntime.make(Store.layer.pipe(Layer.provideMerge(NodeSqlite.layer({filename,readerWorker:true,checkpointWorker:false}))));
}

import {Worker} from 'node:worker_threads';
import {DatabaseSync} from 'node:sqlite';
import {once} from 'node:events';
import {performance} from 'node:perf_hooks';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';

let filename=process.argv[2];const output=process.argv[3];
const holdMs=Number(process.argv[4]??350),rounds=Number(process.argv[5]??6);
const synthetic=!filename||filename==='synthetic';
if(synthetic){
 filename=path.join(fs.mkdtempSync(path.join(os.tmpdir(),'t3-snapshot-contention-')),'statev2.sqlite');
 const seed=ManagedRuntime.make(Store.layer.pipe(Layer.provideMerge(SqlitePersistence.layerFromPath(filename)),Layer.provide(NodeServices.layer)));
 try{await seed.runPromise(Effect.gen(function*(){
  const p=yield*Store.ProjectionStoreV2,at=yield*DateTime.now;
  const threadId='audit:thread';
  yield*p.apply({id:'audit:event:create',type:'thread.created',threadId,occurredAt:at,payload:{id:threadId,projectId:'audit:project',title:'Synthetic audit thread',providerInstanceId:'codex',modelSelection:{instanceId:'codex',model:'audit-model'},runtimeMode:'full-access',interactionMode:'default',branch:null,worktreePath:null,activeProviderThreadId:null,lineage:{parentThreadId:null,relationshipToParent:null,rootThreadId:threadId},forkedFrom:null,createdBy:'user',creationSource:'web',createdAt:at,updatedAt:at,archivedAt:null,deletedAt:null,settledOverride:null,settledAt:null,lastVisitedAt:null}});
  yield*p.apply({id:'audit:event:item',type:'turn-item.updated',threadId,occurredAt:at,payload:{id:'audit:item',type:'command_execution',threadId,runId:null,nodeId:null,providerThreadId:null,providerTurnId:null,nativeItemRef:null,parentItemId:null,ordinal:1,status:'completed',title:'Synthetic command',input:'true',output:'ok',exitCode:0,startedAt:at,completedAt:at,updatedAt:at}});
 }));}finally{await seed.dispose();}
}
const c=new DatabaseSync(filename,{readOnly:true});
const t=c.prepare(`SELECT t.thread_id,count(*) items FROM orchestration_v2_projection_threads t JOIN orchestration_v2_projection_turn_items i USING(thread_id) WHERE t.deleted_at IS NULL GROUP BY t.thread_id ORDER BY items LIMIT 1`).get();c.close();
const runtime=openRuntime(filename);
const results=[];
let sqlRecords=[];
const tracer=Tracer.make({span(o){const s=new Tracer.NativeSpan(o),end=s.end.bind(s);s.end=(at,exit)=>{end(at,exit);if(s.name==='sqlite'||s.name==='sql.transaction')sqlRecords.push({name:s.name,ms:Number(at-s.startTime)/1e6,connection:s.attributes.get('db.connection'),lock_wait_ms:s.attributes.get('db.lock_wait_ms'),query:s.attributes.get('db.query.text')});};return s;}});
const workerCode=`const {parentPort,workerData}=require('node:worker_threads');const{DatabaseSync}=require('node:sqlite');const db=new DatabaseSync(workerData.filename);db.exec('PRAGMA busy_timeout=5000');parentPort.on('message',async m=>{if(m==='hold'){db.exec('BEGIN IMMEDIATE');parentPort.postMessage('held');setTimeout(()=>{db.exec('ROLLBACK');parentPort.postMessage('released');},workerData.holdMs);}if(m==='close'){db.close();parentPort.close();}});parentPort.postMessage('ready');`;
const blocker=new Worker(workerCode,{eval:true,workerData:{filename,holdMs}});
try{
 await once(blocker,'message');
 await runtime.runPromise(Effect.gen(function*(){const sql=yield* SqlClient.SqlClient;yield* sql`PRAGMA busy_timeout=5000`;yield* sql`PRAGMA journal_mode=WAL`;}));
 for(let round=0;round<rounds;round++)for(const mode of round%2?['reader-control','default']:['default','reader-control']){
  const held=once(blocker,'message');blocker.postMessage('hold');const [msg]=await held;if(msg!=='held')throw new Error('Expected held');
  const release=once(blocker,'message');
  const start=performance.now();let heartbeatMs;
  const heartbeat=new Promise(resolve=>setTimeout(()=>{heartbeatMs=performance.now()-start;resolve();},1));
  let op=Effect.gen(function*(){return yield* (yield* Store.ProjectionStoreV2).getThreadSnapshotWindow(t.thread_id,{rowLimit:77,userTurnLimit:10});});
  if(mode==='reader-control')op=NodeSqlite.readOnly(op);
  sqlRecords=[];
  const snap=await runtime.runPromise(op.pipe(Effect.withTracer(tracer)));const snapshotMs=performance.now()-start;
  await heartbeat;await release;
  const row={round,mode,hold_ms:holdMs,synthetic,snapshot_ms:snapshotMs,heartbeat_ms:heartbeatMs,retained_items:snap.projection.turnItems.length,sql:sqlRecords};results.push(row);
  console.log(JSON.stringify({...row,sql:undefined}));
 }
 if(output)fs.writeFileSync(output,JSON.stringify(results,null,2));
}finally{
 blocker.postMessage('close');await once(blocker,'exit');await runtime.dispose();
 if(synthetic)fs.rmSync(path.dirname(filename),{recursive:true,force:true});
}

Tested current main 70e5e384da on Linux, Node 24.21.0, Chrome web via vp run dev, using isolated state. An existing populated stopped thread opened successfully. A larger synthetic projection fixture (100 stopped threads × 300 completed command items, 3 KB output/item; 125 MB SQLite file) also opened successfully in a fresh incognito profile. The sidebar contained 100 fixture rows, but opening it requested the selected conversation's snapshot, rather than a snapshot for every sidebar row.

A browser-issued burst of 100 distinct bounded-snapshot HTTP requests with 6,000 ms abort deadlines returned 100 HTTP 200 responses in 145–1,513 ms. Opening a populated fixture through the UI returned HTTP 200 in 23 ms server-side:

GET /api/orchestration/threads/triage%3A17765%3Athread%3A1/bounded
http.span=23ms
http.status=200

The controlled writer-lock probe in the existing comment is separate evidence; these results do not disprove contention under that condition. The cloudflared/relay path was not verified.

Please provide the current affected build, approximate database size/thread/item counts, the exact UI steps that mount multiple conversations during a cold open, and sanitized request timings showing the bounded requests timing out and falling back. Those details would distinguish the cold-open trigger from the independently demonstrated injected-lock stall. No database contents or credentials are needed.

🤖 Generated by an LLM on behalf of Tomasz Tórz.

Data for the questions above.

Production host: VPS, 8 vCPU, statev2.sqlite 6.06 GiB (0 KiB reclaimable per t3 doctor --dry-run), agents running. Its own traces: http.orchestration.loadThreadSnapshotWindow p50 235 ms, p95 2.4 s, max 4.4 s over 30 min. The slow ones are one sql.transaction with a 675 ms wait before the first read and 20 sequential queries running ~10x slower than in a fast request: CPU contention with the agents, not a query plan. The same queries on an idle machine with a copy of the same data: 29–81 ms for the largest threads (1,000–2,000 items).

Client effect (iOS simulator, same data copy, weak 5G 250 ms / 2 Mbps, server CPU at 10 %, host answering the bounded snapshot after 6.5 s), touch → first feed for never-opened threads:

Thread	6 s budget (main)	20 s budget
14 rows	19.91 s	8.28 s
12 rows	8.36 s	7.84 s
3 rows	6.89 s	7.50 s
3 rows	6.88 s	7.85 s

After the timeout, the socket fallback downloads the thread again, uncompressed on iOS, over the same slow link: cheap for small threads, very expensive for big ones. A longer client budget removes the tail but makes small threads wait for the slow HTTP answer; the server deadline proposed above would avoid both. I have the client change and a harness (sanitized DB copy, shaped Wi-Fi/5G, CPU-throttled host) if you want either.

Additional user report (2026-11-01): loading earlier activity on mobile through the managed relay still fails after more than five reloads.

This is the older-history paging path, rather than just initial bounded-snapshot loading. The user supplied a screenshot showing:

The Load earlier activity control above an inline timeout error.
Already-loaded conversation content remains visible below the error; the thread also shows ongoing work.
The failing request is GET https://<managed-relay-host>/api/orchestration/threads/<thread-id>/history?cursor=<redacted>&deferWork=1&compactIds=1.
The error ends with timed out after 6000ms.
The user reports reloading more than five times without resolving it. This is a user-observed reproduction, not a deterministic test I ran.

Expected: earlier activity loads, or the client provides useful recovery when the history request fails. Actual: older activity remains inaccessible through repeated reloads, and the raw endpoint URL and very long encoded cursor occupy much of the mobile viewport.

Source inspection at fork checkout fd5c10196a08458c3b876e235f8d48836fe1ce20:

packages/client-runtime/src/state/threadHistoryHttp.ts sets DEFAULT_THREAD_HISTORY_TIMEOUT_MS = 6_000 and supplies deferWork=1 and compactIds=1 for ordinary history paging, matching the screenshot.
packages/client-runtime/src/rpc/http.ts applies the request deadline and includes the entire request URL in RemoteEnvironmentAuthTimeoutError.message.
packages/client-runtime/src/state/threads.ts passes that message into history error state through formatHistoryError; a failed HTTP history page returns an error rather than taking the bounded-snapshot loader's socket fallback path.

Read-only inspection of the installed Windows desktop server's rotated server.trace.ndjson.1 also found history requests for the exact thread ID shown in the screenshot. One correlated span chain is:

environment.orchestration.threadHistoryPage  19525.2038 ms
  ThreadManagementService.getThreadHistoryPage  19524.8396 ms
    OrchestratorV2.getThreadHistoryPage  19524.646 ms  Interrupted
      sql.transaction  19524.258 ms
        db.connection=reader
        db.lock_wait_ms=0.006

The interruption stack includes ProjectionStore.readTimelineIndex. Other history service spans in that same rotated log were interrupted after 20345.4362 ms, 22166.5878 ms and 18557.0027 ms; those additional spans have not individually been correlated to the screenshot's thread. A later history service span succeeded in 3513.874 ms. These are observed server span durations, not a benchmark or proof that any specific mobile reload generated a particular span. The common trace ID is not useful for request correlation here; the chain above was correlated by parent/span IDs, and the endpoint span's path supplies the thread ID.

Investigation update

The installed desktop/server version is 0.1.7-nightly.20261101.4016.fork.1, established from the local server-runtime record and desktop version marker. The mobile client version remains unknown. The history-loading implementation was verified directly in the installed server bundle, rather than assumed to match the checkout.

Confirmed expensive path: getThreadHistoryPage invokes ProjectionStore.readTimelineIndex before applying the cursor/page selection. That reconstructs the entire thread timeline, loads its runs and attempts, and queries every local activity row in ordinal order. With deferWork=1, the query also joins nodes and attempts, performs a correlated latest-attempt lookup, and scans dynamic-tool payload text with LIKE checks to classify persistent output. The affected thread has approximately 38,000 activity rows. The SQL has no page limit; the cursor bounds the later selection, not this preparatory read. Fork ancestry can add recursive timeline reads, though that has not been established as a factor for this thread.

The read-only query plan uses the thread/ordinal index and indexed join lookups. This is not evidence of a missing basic thread index or a whole-database table scan; the problem is the amount of full-thread work performed per page.

Request correlation was extended using parent/span IDs. A matching-thread request lasted 22166.7921 ms before interruption, with a reader transaction lasting 22166.2371 ms and reported reader acquisition wait of 0.006 ms. Within it, readTimelineIndex was interrupted after 5036.6243 ms: its records read took 1099.4783 ms and its timeline SQL span took 3936.8103 ms before interruption. The full transaction duration includes cancellation/cleanup; it must not be described as the timeline SELECT's execution time. The matching-thread request that succeeded in 3514.0579 ms spent 3300.1203 ms in readTimelineIndex, including 3004.5822 ms in the timeline SQL span, while page materialization took 127.7366 ms.

Database load matters too. During the slow matching-thread request, other overlapping database spans were also slow, including shell/projection reads and simple node/policy lookups. A standalone read-only execution of the timeline SELECT, using local Node v24.21.0 / SQLite 3.53.4, returned approximately 38,000 rows in about one second. That diagnostic is not the installed server runtime or an end-to-end history-handler reproduction. It establishes that the query is not inevitably slower than the client deadline in isolation; the precise reason for its larger latency inside the busy server remains unresolved. CPU, disk, worker scheduling and transfer overhead have not been isolated, and these traces do not directly measure time executing SQLite's VM separately from worker response time.

Cancellation can retain work: in packages/shared/src/nodeSqliteClient.ts, the reader worker executes synchronous statement.all(...) and posts the result back. The request callback has no cancellation action that interrupts that running SQLite statement. Cancelling the caller therefore does not itself stop an in-progress statement; transaction rollback/cleanup can wait behind it. This is a source-supported mechanism consistent with the much longer outer transaction lifetimes, not proof that each reload created a particular amount of extra load.

Keeping this update on #17765 because it explicitly includes threadHistoryPage and the six-second timeout/cancellation chain. This reproduction is older-history paging on a Windows desktop server with a mobile relay client, rather than the original Linux/web cold-open herd. The original report's single-reader explanation should not be assumed to describe this newer build; the inspected checkout uses a four-reader pool. #24895 is relevant background on costly history reads, but no identical root query has been established between the reports.

The supported diagnosis is full-thread history preparation exceeding the fixed client budget under server/database load, with running SQLite work surviving caller cancellation. A durable fix should investigate bounding the pre-page work and cancellation/recovery together. Raising the timeout alone does not address that work. No fix or deterministic end-to-end reproduction is claimed.

The screenshot contains private conversation content, so this report transcribes the error with relay host, thread ID, and cursor redacted instead of publishing the original image.

Reported on the user's behalf by GPT-6.1 Sol (high reasoning), through Codex in T3 Code.

Synthetic history follow-up — 2026-11-02

Follow-up to #17765 (comment). These experiments concern older-history paging, not the bounded-snapshot socket-fallback path. Related background: #24895.

The original failure did not reproduce at the supplied aggregate size. A larger fixture did cross the six-second boundary under four concurrent history requests, then recovered on the next identical burst. Persistent failure across five or more reloads was not reproduced.

The result supports history-wide preparation as an expensive path, and demonstrates a possible deadline crossing with sufficiently large synthetic history. It does not identify the cause of the extra in-server slowdown observed on Windows, or establish that the managed relay caused it.

Environment and isolation

Starting and final shared checkout: clean main, 96f3bdddc43074bcb2f5d02d1a9b17020c09371f. No application changes, Git integration, PR, or deployment. This comment publishes the diagnostic results and scripts only.
Lazarus T3 container, Linux x86_64, kernel 5.15.0-198-generic; Intel Core i5-11600K, 6 physical/12 logical host CPUs. Container CPU quota 800000/100000 (8 CPUs), memory limit 12 GiB. Host resources are shared. Observed load averages varied around 7–9; no caches or resource limits were changed. No container OOM events occurred. Timing variance must not be interpreted as controlled hardware performance.
Benchmark runtime: Node v24.22.1, built-in SQLite 3.53.4, Effect 4.0.5. This is source execution at the pinned revision, not a measurement of the installed Lazarus binary or of the Windows desktop bundle.
Tracked source was exported to an isolated external work directory; dependencies already installed in another checkout were reused. Package resolution uses the pinned snapshot workspace source and installed third-party modules. No local source modifications were made. The public scripts below default to a separate installed checkout named source/; HISTORY_BENCH_DEPENDENCIES can instead select an existing matching dependency installation.
All state is synthetic and lives outside the shared checkout. No live user databases, conversations, provider logs, attachments, credentials, or secrets were accessed. No full server, real provider, scheduler, integration, relay, or listening socket was started.

Fixture and path exercised

Seed 20261102. The baseline has exactly 38,000 activities, 4,900 runs, 4,900 attempts, 30,000 nodes, and 136,099,170 bytes of activity JSON (136 MB decimal, 129.8 MiB). Its activity counts are 9,800 commands, 6,700 dynamic tools, 5,600 assistant messages, 4,100 checkpoints, 3,500 reasoning items, 3,000 user messages, 2,000 notifications, 1,800 subagents, 1,400 file changes, and 100 system notices. Text is deterministic invented content, interleaved with a seeded shuffle. Tool text reaches 200,000 bytes; file diffs reach 120,000 bytes.

The other fixtures multiply all counts by two/four. Two-times activity payload is about 272 MB; four-times is 545,162,396 bytes. In the original retained scale2/scale4 metadata the counts histogram records base counts; multiply it by scale. The final reusable generator explicitly records both base and scaled counts.

Real migrations 1–76 created WAL databases. All persisted payloads were checked against the real AppThread/Run/RunAttempt/ExecutionNode/TurnItem JSON schemas. PRAGMA integrity_check returned ok after each seed.

Requests go through the real OrchestrationHttp.layer and HttpApi schema/serialization, with deferWork=1&compactIds=1, a real encoded/decoded cursor, and a synthetic authenticated principal carrying the read scope. A narrow synthetic ThreadManagementService port forwards all five arguments to the real ProjectionStore. This bypasses orchestration startup and transcript import, not ProjectionStore's real getThreadHistoryPage, four-reader pool, read-only transaction acquisition, SQL, worker, page selection, materialization, deferred grouping, wire projection, or compact-ID response codec. Valid successful pages contain 282/350/294 rows at baseline/2×/4× and retain hasMoreHistory=true.

The six-second boundary uses AbortController against the in-process web handler. It models client abandonment; it does not invoke mobile's authenticated HTTP client or traverse a network/relay. Measurements include response body reading and successful response validation. Final baseline output additionally separates handler/body from client decode.

Results

The complete request matrix is below, generated from retained JSON results. The scripts recreate detailed worker/Effect timings and standalone query plans. No claim is made that the first process request is disk-cold: fixture creation, migrations, prior runs, and shared host activity influence cache state. “Quiet” means no other benchmark workload, not an idle host.

Final baseline rerun: warm serial requests 0.445–0.549 s, four concurrent requests 1.025–1.386 s, all successful. Earlier baseline runs also succeeded, up to 1.170 s serial. At 2×, quiet requests took 1.111–1.361 s and four concurrent requests 3.053–3.200 s, all successful.

At 4× (152,000 activities, 19,600 runs/attempts, 120,000 nodes), quiet requests took 3.307–3.877 s; paired loads took 4.604–4.942 s. The first four-request burst aborted all four calls: timers fired at 6.068–6.086 s and handler/body settlement followed after another 13.8–16.7 ms. These are actual six-second deadline crossings, with small timer scheduling delay. The in-process HTTP adapter returned its local cancellation status 499; this is not a remote server status observed by mobile. The next identical four-request burst succeeded in 5.430–5.477 s. Thus the benchmark does not demonstrate an accumulating reload/cancellation loop.

Writer comparison correction: the earlier scale2/scale4 runs issued twenty UPDATE statements with the existing timestamp. These are update attempts, not evidence of physical write contention. The final baseline harness was corrected to change both the column and its valid JSON timestamp on every update, and the baseline matrix was rerun successfully. Only that final baseline phase establishes actual concurrent synthetic metadata changes; no throughput or contention conclusion is drawn from it.

Timing boundaries

Baseline warm serial spans: canonical records (runs/attempts and thread) 101–138 ms; full timeline reconstruction 398–492 ms; timeline worker SQLite execution 197–251 ms; worker roundtrip 261–328 ms. Non-empty page materialization calls were under 15 ms. Reader permit waits were 0.016–0.026 ms. These stages nest: do not add them as independent costs.

At 4×/four requests, canonical reads were 1.507–1.821 s and timeline reconstruction 3.964–5.649 s across the two bursts. The first burst's timeline SELECTs spent roughly 2.68–2.74 s executing inside the worker, with roundtrips about 3.10–3.95 s. The reader permit waits across both bursts were only 0.007–0.028 ms. The large main-thread/worker/SQLite differences are observed timings, not proof of CPU, disk, transfer, or scheduler causation. A roundtrip includes worker dispatch, preparation, execution, result serialization/transfer and main-thread scheduling. The execution hook measures synchronous statement.all plus its tiny diagnostic start message; it excludes statement preparation.

Standalone comparisons use the exact captured timeline SELECT with its two bound parameters on each synthetic database, without the Effect service or worker transfer. Repeated baseline samples were 0.541/0.251/0.269 s; 2× 1.122/0.486/0.423 s; 4× 1.182/0.827/0.897 s. They were run sequentially after the corresponding service experiments with no cache clearing. They are not matched simultaneous controls and do not establish why in-service execution was slower. Query plans retain the thread/ordinal index, node/attempt primary-key lookups and indexed correlated latest-attempt lookup; they do not scan the entire database.

Cancellation, separately tested

The milestone probe aborts after the worker announces execution of the timeline SELECT. In the final baseline run, abort occurred about 102 ms after request start; the worker still returned all 38,000 metadata rows. Its statement.all duration was 204 ms and roundtrip 282 ms. HTTP cleanup settled about 284 ms after abort, with the outer request lasting about 386 ms. At 4×, the worker returned all 152,000 rows; execution was 1.036 s, roundtrip 1.339 s, and cleanup about 1.340 s after abort. The next request succeeded at every size.

This directly confirms that caller cancellation did not interrupt the synchronous in-flight SQLite statement. The interrupted timeline Effect span ends before the worker reply; the enclosing transaction/handler includes cleanup waiting. Never label that outer duration “pure SELECT time.” In the actual six-second first burst, cleanup was short because the expensive timeline SELECTs had already returned. Do not conflate that burst with the separate mid-SELECT probe.

Limits and next experiment

Single synthetic thread, no inherited fork chain, one completed attempt per run, no live activity, no pending provider work, and no private event-log reconstruction. All fixture nodes are valid, but activities reference root nodes; the extra child nodes are present without being joined by history items. This simplifies node lookup locality relative to the aggregate original shape.
Synthetic payload content, ordering and user-turn boundaries cannot match the private thread. No message table hydration, mobile rendering, managed-relay transport, provider ingestion, or production background services were exercised. Small metadata writes are not a production ingest simulation.
Benchmark-only instrumentation adds execution-start messages and trace collection. Shared-host load and cache conditions were not controlled. This is a reusable diagnostic, not a performance regression test or deterministic assertion that 4× always times out.
Baseline-sized persistent timeouts and failed reloads remain unexplained. The smallest next local experiment is to distribute item node references over the existing 30,000 valid nodes (preserving root/run/attempt relationships) and rerun the baseline. A subsequent matched baseline run on Windows using the same synthetic fixture would distinguish environment differences without accessing private data. A relay/mobile experiment is needed to test that additional boundary.

All scripts and workers exited. The repository remained clean at the starting revision. No repo-wide checks were run: validation was fixture schema/integrity checks, real HTTP success-codec validation, timing/row-count checks, query plans, abort/recovery behavior, and a final rerun of the reusable harness at baseline.

Source references

The measured revision is a public personal-fork commit, not upstream main:

The timeline plan in all three synthetic databases was:

SEARCH item USING INDEX orchestration_v2_projection_turn_items_thread_ordinal_idx (thread_id=?)
SEARCH node USING INDEX sqlite_autoindex_orchestration_v2_projection_nodes_1 (node_id=?) LEFT-JOIN
SEARCH attempt USING INDEX sqlite_autoindex_orchestration_v2_projection_run_attempts_1 (attempt_id=?) LEFT-JOIN
CORRELATED SCALAR SUBQUERY
SEARCH orchestration_v2_projection_run_attempts USING INDEX orchestration_v2_projection_run_attempts_run_ordinal_idx (run_id=?)

Complete measured request matrix

Seconds; excludes the explicit mid-query cancellation probe.

Fixture	Phase	Count	Successful	Deadline aborts	Observed duration
38,000 activities	Quiet first process request	1	1	0	0.963–0.963 s
38,000 activities	Quiet subsequent	3	3	0	0.445–0.549 s
38,000 activities	Two concurrent, three bursts	6	6	0	0.655–0.978 s
38,000 activities	Four concurrent, first burst	4	4	0	1.327–1.386 s
38,000 activities	Four concurrent, second burst	4	4	0	1.025–1.071 s
38,000 activities	Two plus twenty metadata changes	2	2	0	0.634–0.643 s
38,000 activities	After mid-query cancellation	1	1	0	0.425–0.425 s
76,000 activities	Quiet first process request	1	1	0	1.359–1.359 s
76,000 activities	Quiet subsequent	3	3	0	1.111–1.361 s
76,000 activities	Two concurrent, three bursts	6	6	0	1.779–2.115 s
76,000 activities	Four concurrent, first burst	4	4	0	3.053–3.092 s
76,000 activities	Four concurrent, second burst	4	4	0	3.145–3.200 s
76,000 activities	Two plus twenty unchanged-value UPDATE attempts	2	2	0	2.515–2.529 s
76,000 activities	After mid-query cancellation	1	1	0	1.108–1.108 s
152,000 activities	Quiet first process request	1	1	0	3.568–3.568 s
152,000 activities	Quiet subsequent	3	3	0	3.307–3.877 s
152,000 activities	Two concurrent, three bursts	6	6	0	4.604–4.942 s
152,000 activities	Four concurrent, first burst	4	0	4	6.084–6.100 s
152,000 activities	Four concurrent, second burst	4	4	0	5.430–5.477 s
152,000 activities	Two plus twenty unchanged-value UPDATE attempts	2	2	0	2.937–2.952 s
152,000 activities	After mid-query cancellation	1	1	0	2.966–2.966 s

Runnable scripts

Save the named files below together in a new external work directory. Linux and Node 24.22.1 were tested. The cgroup reads are Linux/container diagnostics; this published version is not a portable Windows harness. No database download is needed: every fixture is generated from invented data using real migrations and payload schemas.

Use a separate source checkout pinned to the measured commit, with its normal locked development dependencies installed:

# Run inside the new external work directory containing the saved scripts.
git clone --filter=blob:none --no-checkout https://github.com/Alb11747/t3code.git source
git -C source checkout --detach 96f3bdddc43074bcb2f5d02d1a9b17020c09371f
(cd source && vp i)
node link-dependencies.mjs

node --import ./instrument.mjs ./reproduce.mjs > baseline-run.log 2>&1
python3 summarize.py baseline > baseline/timing-summary.log
node raw-comparison.mjs baseline

# Run sequentially, not together. No phase exceeds four concurrent history loads.
HISTORY_BENCH_SCALE=2 HISTORY_BENCH_OUT="$PWD/scale2" node --import ./instrument.mjs ./reproduce.mjs > scale2-run.log 2>&1
python3 summarize.py scale2 > scale2/timing-summary.log
node raw-comparison.mjs scale2

HISTORY_BENCH_SCALE=4 HISTORY_BENCH_OUT="$PWD/scale4" node --import ./instrument.mjs ./reproduce.mjs > scale4-run.log 2>&1
python3 summarize.py scale4 > scale4/timing-summary.log
node raw-comparison.mjs scale4

node verify-artifacts.mjs
python3 make-measurements.py

Alternatively, export HISTORY_BENCH_DEPENDENCIES=/path/to/an/already-installed-matching-checkout before linking/running. The recorded run used Effect and @effect/platform-node 4.0.5. Do not point any script at live user state. HISTORY_BENCH_OUT must be a child of this external script directory, outside source/; the database filename is always synthetic.sqlite. Existing fixtures require matching seed/scale/revision metadata. Choose a new child directory to regenerate instead of replacing another database.

The initial baseline database is approximately 191 MiB; all three databases together are approximately 1.3 GiB, excluding transient WAL/temporary space and checkout dependencies. Seed/integrity work can briefly be synchronous. These are bounded diagnostics for a host with available resources, not a stress test; no resource limits or OS caches are modified.

instrument.mjs changes the reader worker source in memory through a Node load hook; it does not edit product code. It adds a start milestone and synchronous SQLite duration, observes actual Worker request/reply timings, and retains Effect spans. The start message itself is included in the recorded SQLite duration, preparation is excluded, and roundtrip minus execution must not be called pure transfer time. The schema/HTTP implementation remains the pinned source. Each request has a six-second AbortController boundary; the mid-query probe aborts at the worker's timeline-execution milestone. Bursts await HTTP cleanup before the next burst starts. The diagnostic deliberately does not implement the authenticated mobile client or relay.

The scripts below include two publication corrections relative to the first 2×/4× run: metadata histograms explicitly distinguish base/scaled counts, and the writer workload changes the JSON/column timestamp on every UPDATE. The final baseline was rerun with the corrected writer; the old 2×/4× writer phase is explicitly labeled unchanged-value UPDATE attempts in the matrix. Default dependency paths were changed to source/ for public reuse; measured fixture contents and timing logic were otherwise preserved. The verification helper also checks the final changed writer timestamp at baseline.

Output: fixture.json, synthetic.sqlite, results.json (per-request results, worker records, complete captured SQL, Effect spans), summary.json, raw-comparison.json, measurements.md, and optional comparison.html. Successful bodies must decode with the actual compact-ID history codec. Unexpected ordinary request errors fail the run; deadline aborts are measured outcomes. The last outcome must show recovery after the explicit cancellation probe. verify-artifacts.mjs checks actual database row counts, response statuses/compact IDs, and the changed writer timestamp.

The optional chart helper makes a standalone local HTML comparison from measured results; it starts no server. No scripts instantiate real providers/schedulers/integrations, read a user database, or start a network listener. Scopes dispose HTTP resources and SQLite reader/checkpoint workers when the finite run completes. Keep/delete the external fixture directory according to your normal local cleanup workflow.

instrument.mjs — complete file

import { registerHooks } from 'node:module';
import { Worker } from 'node:worker_threads';
import fs from 'node:fs';
import path from 'node:path';
import { pathToFileURL, fileURLToPath } from 'node:url';
const work = path.dirname(fileURLToPath(import.meta.url));
const sourceRoot = path.join(work, 'source');
const dependencyRoot = process.env.HISTORY_BENCH_DEPENDENCIES ?? path.join(work, 'source');
const packages = new Map();
for (const dir of fs.readdirSync(path.join(sourceRoot, 'packages'))) {
  const packagePath = path.join(sourceRoot, 'packages', dir, 'package.json');
  if (fs.existsSync(packagePath)) { const json = JSON.parse(fs.readFileSync(packagePath, 'utf8')); packages.set(json.name, { dir: path.dirname(packagePath), json }); }
}
globalThis.historyBench = { workers: [], spans: [], phase: 'startup' };
registerHooks({
  resolve(specifier, context, nextResolve) {
    if (specifier.startsWith('.') || specifier.startsWith('/') || specifier.includes(':')) return nextResolve(specifier, context);
    const parts = specifier.split('/');
    const name = specifier.startsWith('@') ? parts.slice(0, 2).join('/') : parts[0];
    const pkg = packages.get(name);
    if (pkg) {
      const subpath = specifier.slice(name.length);
      let target = pkg.json.exports?.[subpath ? '.' + subpath : '.'];
      if (!target && typeof pkg.json.exports === 'string') target = pkg.json.exports;
      if (!target && !subpath) target = pkg.json.main;
      if (typeof target === 'object') target = target.import ?? target.default ?? target.types;
      if (!target) throw new Error(`Unresolved workspace export ${specifier}`);
      return nextResolve(pathToFileURL(path.join(pkg.dir, target)).href, context);
    }
    if (!context.parentURL?.startsWith(pathToFileURL(work).href + '/')) return nextResolve(specifier, context);
    let parentURL = pathToFileURL(path.join(dependencyRoot, 'apps/server/dependency-anchor.mjs')).href;
    if (context.parentURL?.startsWith(pathToFileURL(sourceRoot).href + '/')) {
      parentURL = context.parentURL.replace(pathToFileURL(sourceRoot).href, pathToFileURL(dependencyRoot).href);
    }
    return nextResolve(specifier, { ...context, parentURL });
  },
  load(url, context, nextLoad) {
    const result = nextLoad(url, context);
    if (!url.endsWith('/packages/shared/src/nodeSqliteClient.ts')) return result;
    const source = String(result.source);
    const before = 'parentPort.postMessage({ id, rows: statement.all(...params) });';
    if (!source.includes(before)) throw new Error('SQLite instrumentation anchor changed');
    return { ...result, source: source.replace(before, `const started = performance.now();
      parentPort.postMessage({ historyBenchStarted: id });
      const rows = statement.all(...params);
      const sqliteMs = performance.now() - started;
      parentPort.postMessage({ id, rows, historyBenchSqliteMs: sqliteMs });`) };
  },
});
const pending = new WeakMap();
const post = Worker.prototype.postMessage;
Worker.prototype.postMessage = function (message, ...rest) {
  if (typeof message?.sql === 'string') {
    let requests = pending.get(this);
    if (!requests) pending.set(this, requests = new Map());
    requests.set(message.id, { start: performance.now(), phase: globalThis.historyBench.phase,
      sql: message.sql, worker: this.threadId });
  }
  return post.call(this, message, ...rest);
};
const emit = Worker.prototype.emit;
Worker.prototype.emit = function (event, message, ...rest) {
  const requests = pending.get(this);
  if (event === 'message' && message?.historyBenchStarted !== undefined && requests?.has(message.historyBenchStarted)) {
    globalThis.historyBench.onSqliteStart?.(requests.get(message.historyBenchStarted));
  }
  if (event === 'message' && requests?.has(message?.id)) {
    const request = requests.get(message.id);
    requests.delete(message.id);
    globalThis.historyBench.workers.push({ ...request, end: performance.now(),
      roundtripMs: performance.now() - request.start,
      sqliteMs: message.historyBenchSqliteMs ?? null, rows: message.rows?.length ?? null });
  }
  return emit.call(this, event, message, ...rest);
};

reproduce.mjs — complete file

import fs from 'node:fs';
import path from 'node:path';
import { pathToFileURL, fileURLToPath } from 'node:url';
import { DatabaseSync } from 'node:sqlite';
import * as Effect from 'effect/Effect';
import * as Layer from 'effect/Layer';
import * as Schema from 'effect/Schema';
import * as Tracer from 'effect/Tracer';
import * as SqlClient from 'effect/sql/SqlClient';
import * as NodeServices from '@effect/platform-node/NodeServices';
import * as NodeHttpPlatform from '@effect/platform-node/NodeHttpPlatform';
import * as HttpApi from 'effect/http-api/HttpApi';
import * as HttpApiBuilder from 'effect/http-api/HttpApiBuilder';
import { Etag, HttpRouter } from 'effect/http';
import * as C from '@t3tools/contracts';

const work = path.dirname(fileURLToPath(import.meta.url));
const repo = path.resolve(process.env.HISTORY_BENCH_REPO ?? path.join(work, 'source'));
const out = path.resolve(process.env.HISTORY_BENCH_OUT ?? path.join(work, 'baseline'));
if (!out.startsWith(work + path.sep) || out.startsWith(repo + path.sep)) throw new Error('Output must be a child of the external benchmark directory');
fs.mkdirSync(out, { recursive: true });
const load = (relative) => import(pathToFileURL(path.join(repo, relative)).href);
const [Persistence, Projection, OrchestrationHttp, Management, Environment, Events, Projects, Enrichment, Paging, NodeSqlite] = await Promise.all([
  'apps/server/src/persistence/Sqlite.ts', 'apps/server/src/orchestration-v2/ProjectionStore.ts',
  'apps/server/src/orchestration-v2/http.ts', 'apps/server/src/orchestration-v2/ThreadManagementService.ts',
  'apps/server/src/environment/ServerEnvironment.ts', 'apps/server/src/persistence/OrchestrationEventStore.ts',
  'apps/server/src/orchestration-v2/ProjectStore.ts', 'apps/server/src/project/ProjectEnrichmentService.ts',
  'apps/server/src/orchestration-v2/threadHistoryPaging.ts', 'packages/shared/src/nodeSqliteClient.ts',
].map(load));
const dbPath = path.join(out, 'synthetic.sqlite');
const seed = 20261102;
const scale = Number(process.env.HISTORY_BENCH_SCALE ?? 1);
if (![1, 2, 4].includes(scale)) throw new Error('Allowed scale: 1, 2, 4');
const threadId = 'synthetic-thread-20261102';
const at = '2026-01-01T00:00:00.000Z';
const meta = { seed, scale, revision: process.env.HISTORY_BENCH_REVISION ?? '96f3bdddc43074bcb2f5d02d1a9b17020c09371f',
  node: process.version, sqlite: process.versions.sqlite, pid: process.pid, dbPath, started: new Date().toISOString(),
  cpuMax: fs.readFileSync('/sys/fs/cgroup/cpu.max', 'utf8').trim(),
  memoryMax: fs.readFileSync('/sys/fs/cgroup/memory.max', 'utf8').trim(),
  loadavg: fs.readFileSync('/proc/loadavg', 'utf8').trim() };
const persistence = Persistence.layerFromPath(dbPath).pipe(Layer.provide(NodeServices.layer));
if (!fs.existsSync(dbPath)) {
  await Effect.runPromise(Effect.void.pipe(Effect.provide(persistence)));
  const db = new DatabaseSync(dbPath);
  db.exec('PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL; BEGIN IMMEDIATE');
  const prepared = new Map();
  const insert = (table, row) => {
    const names = Object.keys(row);
    const query = `INSERT INTO ${table} (${names.join(',')}) VALUES (${names.map(() => '?').join(',')})`;
    let statement = prepared.get(query);
    if (!statement) prepared.set(query, statement = db.prepare(query));
    statement.run(...Object.values(row));
  };
  const validate = Object.fromEntries(['AppThread', 'Run', 'RunAttempt', 'ExecutionNode', 'TurnItem'].map(name =>
    [name, Schema.decodeUnknownSync(C[`OrchestrationV2${name}Json`])]));
  const payload = (kind, value) => { validate[kind](value); return JSON.stringify(value); };
  const thread = { id: threadId, projectId: 'synthetic-project', title: 'Invented history benchmark',
    createdBy: 'user', creationSource: 'web', providerInstanceId: 'codex', modelSelection: { instanceId: 'codex', model: 'synthetic-unused' },
    runtimeMode: 'full-access', interactionMode: 'default', branch: null, worktreePath: null,
    activeProviderThreadId: null, historyOrigin: 'native', lineage: { parentThreadId: null, relationshipToParent: null, rootThreadId: threadId },
    forkedFrom: null, createdAt: at, updatedAt: at, archivedAt: null, settledOverride: null, settledAt: null, lastVisitedAt: null, deletedAt: null };
  insert('orchestration_v2_projection_threads', { thread_id: threadId, project_id: thread.projectId, title: thread.title,
    default_provider: 'codex', provider_instance_id: 'codex', runtime_mode: thread.runtimeMode, interaction_mode: thread.interactionMode,
    active_provider_thread_id: null, created_at: at, updated_at: at, archived_at: null, deleted_at: null, payload_json: payload('AppThread', thread) });
  const runs = 4900 * scale, nodes = 30000 * scale;
  for (let i = 0; i < runs; i++) {
    const run = { id: `run-${i}`, threadId, ordinal: i + 1, providerInstanceId: 'codex', modelSelection: thread.modelSelection,
      providerThreadId: 'synthetic-provider-thread', userMessageId: `message-${i}`, rootNodeId: `node-${i}`, activeAttemptId: `attempt-${i}`,
      status: 'completed', requestedAt: at, startedAt: at, completedAt: at, checkpointId: null, contextHandoffId: null };
    insert('orchestration_v2_projection_runs', { run_id: run.id, thread_id: threadId, ordinal: i + 1, provider: 'codex', provider_instance_id: 'codex',
      provider_thread_id: run.providerThreadId, status: run.status, requested_at: at, completed_at: at, payload_json: payload('Run', run) });
    const attempt = { id: `attempt-${i}`, runId: run.id, attemptOrdinal: 1, rootNodeId: run.rootNodeId, providerInstanceId: 'codex',
      providerThreadId: run.providerThreadId, providerTurnId: `provider-turn-${i}`, reason: 'initial', status: 'completed', startedAt: at, completedAt: at };
    insert('orchestration_v2_projection_run_attempts', { attempt_id: attempt.id, thread_id: threadId, run_id: run.id, attempt_ordinal: 1,
      root_node_id: attempt.rootNodeId, provider: 'codex', provider_instance_id: 'codex', provider_thread_id: attempt.providerThreadId,
      provider_turn_id: attempt.providerTurnId, status: attempt.status, payload_json: payload('RunAttempt', attempt) });
  }
  for (let i = 0; i < nodes; i++) {
    const r = i % runs;
    const node = { id: `node-${i}`, threadId, runId: `run-${r}`, parentNodeId: i < runs ? null : `node-${r}`, rootNodeId: `node-${r}`,
      kind: i < runs ? 'root_turn' : 'tool_call', status: 'completed', countsForRun: true,
      providerThreadId: 'synthetic-provider-thread', providerTurnId: `provider-turn-${r}`, nativeItemRef: null,
      runtimeRequestId: null, checkpointScopeId: null, startedAt: at, completedAt: at };
    insert('orchestration_v2_projection_nodes', { node_id: node.id, thread_id: threadId, run_id: node.runId, parent_node_id: node.parentNodeId,
      root_node_id: node.rootNodeId, kind: node.kind, status: node.status, provider_thread_id: node.providerThreadId, provider_turn_id: node.providerTurnId,
      runtime_request_id: null, checkpoint_scope_id: null, started_at: at, completed_at: at, payload_json: payload('ExecutionNode', node) });
  }
  const counts = { command_execution: 9800, dynamic_tool: 6700, assistant_message: 5600, checkpoint: 4100, reasoning: 3500,
    user_message: 3000, notification: 2000, subagent: 1800, file_change: 1400, system_notice: 100 };
  // Seeded interleaving prevents grouping whole item types into artificial timeline blocks.
  const types = Object.entries(counts).flatMap(([type, count]) => Array(count * scale).fill(type));
  let rng = seed;
  for (let i = types.length - 1; i > 0; i--) { rng = (Math.imul(rng, 1664525) + 1013904223) >>> 0;
    const j = rng % (i + 1); [types[i], types[j]] = [types[j], types[i]]; }
  const text = (n) => 'Synthetic diagnostic text only. '.repeat(Math.ceil(n / 32)).padEnd(n, 's').slice(0, n);
  let bytes = 0;
  for (let i = 0; i < types.length; i++) {
    const type = types[i], r = Math.min(runs - 1, Math.floor(i * runs / types.length));
    // Match each item's node to its run so correlated latest-attempt lookups succeed.
    const nodeId = `node-${r}`;
    const item = { id: `item-${String(i).padStart(7, '0')}`, threadId, runId: `run-${r}`, nodeId,
      providerThreadId: 'synthetic-provider-thread', providerTurnId: `provider-turn-${r}`, nativeItemRef: null,
      parentItemId: null, ordinal: i, status: 'completed', title: 'Synthetic activity', startedAt: at, completedAt: at, updatedAt: at, type };
    const n = i % 97 === 0 ? 200000 : 3500;
    if (type === 'command_execution') Object.assign(item, { input: 'echo synthetic', output: text(n), exitCode: 0 });
    if (type === 'dynamic_tool') Object.assign(item, { toolName: i % 31 === 0 ? 'delegate_task' : 'synthetic_lookup', input: { synthetic: true }, output: { text: text(n) } });
    if (type === 'assistant_message') Object.assign(item, { messageId: `message-assistant-${i}`, text: text(1200), streaming: false });
    if (type === 'user_message') Object.assign(item, { messageId: `message-user-${i}`, createdBy: 'user', creationSource: 'web', inputIntent: 'turn_start', text: text(500), attachments: [] });
    if (type === 'reasoning') Object.assign(item, { text: text(1800), streaming: false });
    if (type === 'checkpoint') Object.assign(item, { checkpointId: `checkpoint-${i}`, scopeId: `scope-${i}`, files: [] });
    if (type === 'notification') Object.assign(item, { source: { kind: 'system' }, outcome: 'completed', summary: 'Synthetic notification' });
    if (type === 'subagent') Object.assign(item, { subagentId: `synthetic-child-${i}`, origin: 'app_owned', driver: 'codex', providerInstanceId: 'codex', childThreadId: null, prompt: text(200), result: text(1000) });
    if (type === 'file_change') Object.assign(item, { fileName: 'synthetic.txt', additions: 1, deletions: 1, diffStr: text(i % 97 === 0 ? 120000 : 5000) });
    if (type === 'system_notice') Object.assign(item, { message: 'Synthetic notice' });
    const json = payload('TurnItem', item); bytes += Buffer.byteLength(json);
    insert('orchestration_v2_projection_turn_items', { turn_item_id: item.id, thread_id: threadId, run_id: item.runId, node_id: nodeId,
      provider_thread_id: item.providerThreadId, provider_turn_id: item.providerTurnId, parent_item_id: null, ordinal: i, type, status: item.status, updated_at: at, payload_json: json });
  }
  db.exec('COMMIT; PRAGMA wal_checkpoint(TRUNCATE)');
  meta.shape = { items: types.length, runs, attempts: runs, nodes, payloadBytes: bytes, baseCounts: counts,
    counts: Object.fromEntries(Object.entries(counts).map(([type, count]) => [type, count * scale])) };
  meta.integrity = db.prepare('PRAGMA integrity_check').get();
  db.close();
  fs.writeFileSync(path.join(out, 'fixture.json'), JSON.stringify(meta, null, 2));
} else {
  const fixture = JSON.parse(fs.readFileSync(path.join(out, 'fixture.json'), 'utf8'));
  if (fixture.seed !== seed || fixture.scale !== scale || fixture.revision !== meta.revision) throw new Error('Existing fixture metadata differs');
  meta.shape = fixture.shape;
}

const bench = globalThis.historyBench;
if (!bench) throw new Error('Run with --import ./instrument.mjs');
const tracer = Tracer.make({ span(options) {
  const span = new Tracer.NativeSpan(options), end = span.end.bind(span), start = performance.now(), phase = bench.phase;
  span.end = (time, exit) => { end(time, exit); bench.spans.push({ name: span.name, phase, start, end: performance.now(), ms: performance.now() - start,
    attributes: Object.fromEntries(span.attributes), exit: exit._tag, spanId: span.spanId,
    parent: span.parent._tag === 'Some' ? span.parent.value.spanId : null }); };
  return span;
} });
const storeLayer = Projection.layer.pipe(Layer.provideMerge(persistence));
const results = [];
await Effect.runPromise(Effect.scoped(Effect.gen(function* () {
  const store = yield* Projection.ProjectionStoreV2;
  const sql = yield* SqlClient.SqlClient;
  const context = yield* Effect.context();
  const management = Layer.mock(Management.ThreadManagementService)({
    getThreadHistoryPage: (...args) => store.getThreadHistoryPage(...args),
  });
  class Api extends HttpApi.make('environment').add(C.EnvironmentHttpApi.groups.orchestration) {}
  const auth = Layer.succeed(C.EnvironmentAuthenticatedAuth, effect => effect.pipe(Effect.provideService(C.EnvironmentAuthenticatedPrincipal,
    { sessionId: 'synthetic-session', subject: 'synthetic-subject', method: 'browser-session-cookie', scopes: new Set([C.AuthOrchestrationReadScope]) })));
  const dependencies = Layer.mergeAll(management, Layer.succeed(Environment.ServerEnvironmentIdentity, { getEnvironmentId: Effect.succeed('synthetic-environment') }),
    Layer.mock(Events.OrchestrationEventStore)({}), Layer.mock(Projects.ProjectStoreV2)({}), Layer.mock(Enrichment.ProjectEnrichmentService)({}));
  const appLayer = HttpApiBuilder.layer(Api).pipe(Layer.provide(OrchestrationHttp.layer), Layer.provide(dependencies), Layer.provide(auth),
    Layer.provide(Layer.succeedContext(context)), Layer.provide(NodeHttpPlatform.layer), Layer.provide(Etag.layerWeak), Layer.provide(NodeServices.layer),
    Layer.provide(Layer.succeed(Tracer.Tracer, tracer)));
  const web = HttpRouter.toWebHandler(appLayer, { disableLogger: true });
  yield* Effect.addFinalizer(() => Effect.promise(() => web.dispose()));
  const cursor = Paging.encodeThreadHistoryCursor({ snapshotSequence: 0, sourceThreadId: threadId,
    sourceItemId: `item-${String(meta.shape.items - 100).padStart(7, '0')}`, position: meta.shape.items - 100 });
  Paging.decodeThreadHistoryCursor(cursor);
  const request = async (label, deadline = 6000, cancelTimeline = false) => {
    const started = performance.now();
    const controller = new AbortController();
    let deadlineAt = null;
    if (cancelTimeline) bench.onSqliteStart = request => {
      if (!request.sql.includes('SELECT item.turn_item_id, item.run_id')) return;
      bench.onSqliteStart = null;
      deadlineAt = performance.now(); controller.abort();
    };
    const timer = setTimeout(() => { deadlineAt = performance.now(); controller.abort(); }, deadline);
    try {
      const response = await web.handler(new Request(`http://synthetic.test/api/orchestration/threads/${threadId}/history?cursor=${encodeURIComponent(cursor)}&deferWork=1&compactIds=1`,
        { signal: controller.signal, headers: { [C.ORCHESTRATION_PROTOCOL_HEADER]: String(C.ORCHESTRATION_PROTOCOL_VERSION) } }));
      const body = await response.text();
      const handlerAndBodyMs = performance.now() - started;
      if (controller.signal.aborted) {
        const row = { label, ms: performance.now() - started, status: response.status, aborted: true,
          deadlineMs: deadlineAt - started, cleanupMs: performance.now() - deadlineAt };
        results.push(row); console.log(JSON.stringify(row)); return;
      }
      const wire = JSON.parse(body);
      if (response.status !== 200) console.log(body.slice(0, 1000));
      const decoded = response.status === 200 ? Schema.decodeUnknownSync(Schema.fromJsonString(Schema.toCodecJson(C.OrchestrationV2ThreadHistoryPage)))(body) : null;
      const row = { label, ms: performance.now() - started, status: response.status, bytes: Buffer.byteLength(body), compactIds: Array.isArray(wire.ids),
        handlerAndBodyMs, clientDecodeMs: performance.now() - started - handlerAndBodyMs,
        items: decoded?.items.length, hasMore: decoded?.hasMoreHistory, deadlineMs: deadlineAt === null ? null : deadlineAt - started,
        cleanupMs: deadlineAt === null ? null : performance.now() - deadlineAt };
      results.push(row); console.log(JSON.stringify(row));
    } catch (error) {
      const row = { label, ms: performance.now() - started, error: String(error), deadlineMs: deadlineAt === null ? null : deadlineAt - started,
        cleanupMs: deadlineAt === null ? null : performance.now() - deadlineAt };
      results.push(row); console.log(JSON.stringify(row));
    } finally { clearTimeout(timer); if (cancelTimeline) bench.onSqliteStart = null; }
  };
  yield* Effect.promise(async () => {
    bench.phase = 'quiet-first-process-cache'; await request('quiet-1');
    bench.phase = 'quiet-warm'; for (let i = 2; i <= 4; i++) await request(`quiet-${i}`);
    bench.phase = 'two-concurrent'; for (let round = 1; round <= 3; round++) await Promise.all([request(`two-${round}-a`), request(`two-${round}-b`)]);
    bench.phase = 'four-concurrent'; for (let round = 1; round <= 2; round++) await Promise.all(Array.from({ length: 4 }, (_, i) => request(`four-${round}-${i}`)));
  });
  // One bounded synthetic writer plus two history loads. No production connection exists.
  bench.phase = 'two-with-writes';
  yield* Effect.all([
    Effect.promise(() => Promise.all([request('writes-a'), request('writes-b')])),
    Effect.forEach(Array.from({ length: 20 }), (_, i) => {
      const writtenAt = `2026-01-01T00:00:${String(i + 1).padStart(2, '0')}.000Z`;
      return sql`UPDATE orchestration_v2_projection_threads SET updated_at = ${writtenAt},
        payload_json = json_set(payload_json, '$.updatedAt', ${writtenAt}) WHERE thread_id = ${threadId}`;
    }),
  ], { concurrency: 2 });
  // The worker announces execution before statement.all; cancellation follows that milestone.
  bench.phase = 'cancel-timeline-milestone';
  yield* Effect.promise(() => request('cancel-probe', 6000, true));
  bench.phase = 'post-cancel';
  yield* Effect.promise(() => request('after-cancel'));
})).pipe(Effect.provide(storeLayer), Effect.provideService(Tracer.Tracer, tracer)));
fs.writeFileSync(path.join(out, 'results.json'), JSON.stringify({ meta, results, workers: bench.workers, spans: bench.spans,
  finished: new Date().toISOString(), loadavg: fs.readFileSync('/proc/loadavg', 'utf8').trim() }, null, 2));
console.log(`Saved ${path.join(out, 'results.json')}`);
if (results.some(row => row.label !== 'cancel-probe' && row.status !== 200 && !row.aborted)) process.exitCode = 1;

link-dependencies.mjs — complete file

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const work = path.dirname(fileURLToPath(import.meta.url));
const dependencyRoot = process.env.HISTORY_BENCH_DEPENDENCIES ?? path.join(work, 'source');
for (const area of ['packages', 'apps']) {
  for (const name of fs.readdirSync(path.join(work, 'source', area))) {
    const target = path.join(dependencyRoot, area, name, 'node_modules');
    const destination = path.join(work, 'source', area, name, 'node_modules');
    if (fs.existsSync(target) && !fs.existsSync(destination)) fs.symlinkSync(target, destination);
  }
}

summarize.py — complete file

import json
import pathlib
import statistics
import sys

root = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else pathlib.Path(__file__).parent / 'baseline')
data = json.loads((root / 'results.json').read_text())
summary = {'meta': data['meta'], 'requests': data['results'], 'phases': {}}
for phase in dict.fromkeys(row['phase'] for row in data['spans']):
    if phase == 'startup':
        continue
    spans = [row for row in data['spans'] if row['phase'] == phase]
    workers = [row for row in data['workers'] if row['phase'] == phase]
    groups = {}
    for name in ['ProjectionStore.readCanonicalProjection', 'ProjectionStore.readTimelineIndex',
                 'ProjectionStore.materializeHistoryRows', 'environment.orchestration.threadHistoryPage', 'sql.transaction']:
        times = [row['ms'] for row in spans if row['name'] == name]
        if times:
            groups[name] = {'n': len(times), 'min': min(times), 'median': statistics.median(times), 'max': max(times)}
    timeline = [row for row in workers if 'SELECT item.turn_item_id, item.run_id' in row['sql']]
    groups['timelineWorker'] = [{key: row[key] for key in ['worker', 'rows', 'sqliteMs', 'roundtripMs', 'start', 'end']} for row in timeline]
    groups['lockWaitMs'] = [row['attributes']['db.lock_wait_ms'] for row in spans if 'db.lock_wait_ms' in row['attributes']]
    summary['phases'][phase] = groups
(root / 'summary.json').write_text(json.dumps(summary, indent=2) + '\n', encoding='utf-8', newline='\n')
for phase, groups in summary['phases'].items():
    print(phase, json.dumps(groups))

raw-comparison.mjs — complete file

import fs from 'node:fs';
import path from 'node:path';
import { DatabaseSync } from 'node:sqlite';
const out = path.resolve(process.argv[2]);
const results = JSON.parse(fs.readFileSync(path.join(out, 'results.json'), 'utf8'));
const query = results.workers.find(row => row.sql.includes('SELECT item.turn_item_id, item.run_id')).sql;
const db = new DatabaseSync(path.join(out, 'synthetic.sqlite'), { readOnly: true });
const params = ['%t3McpApp%', 'synthetic-thread-20261102'];
const plan = db.prepare('EXPLAIN QUERY PLAN ' + query).all(...params);
const samples = [];
for (let i = 0; i < 3; i++) {
  const start = performance.now();
  const rows = db.prepare(query).all(...params);
  if (rows.length !== results.meta.shape.items) throw new Error('Raw row count mismatch');
  samples.push({ ms: performance.now() - start, rows: rows.length });
}
db.close();
const report = { label: 'Standalone read-only, after service runs; OS cache not cleared', plan, samples, query };
fs.writeFileSync(path.join(out, 'raw-comparison.json'), JSON.stringify(report, null, 2));
console.log(JSON.stringify({ samples, plan }));

verify-artifacts.mjs — complete file

import fs from 'node:fs';
import path from 'node:path';
import assert from 'node:assert/strict';
import { fileURLToPath } from 'node:url';
import { DatabaseSync } from 'node:sqlite';
const work = path.dirname(fileURLToPath(import.meta.url));
for (const name of ['baseline', 'scale2', 'scale4']) {
  const dir = path.join(work, name);
  const data = JSON.parse(fs.readFileSync(path.join(dir, 'results.json'), 'utf8'));
  const db = new DatabaseSync(path.join(dir, 'synthetic.sqlite'), { readOnly: true });
  for (const [table, expected] of [['turn_items', data.meta.shape.items], ['runs', data.meta.shape.runs],
    ['run_attempts', data.meta.shape.attempts], ['nodes', data.meta.shape.nodes]]) {
    assert.equal(db.prepare(`SELECT count(*) AS n FROM orchestration_v2_projection_${table}`).get().n, expected);
  }
  for (const row of data.results) {
    if (row.aborted) assert.equal(row.status, 499);
    else { assert.equal(row.status, 200); assert.equal(row.compactIds, true); assert.ok(row.items > 0); }
  }
  if (name === 'baseline') {
    const row = db.prepare('SELECT updated_at, json_extract(payload_json, \'$.updatedAt\') AS payload_time FROM orchestration_v2_projection_threads').get();
    assert.equal(row.updated_at, '2026-01-01T00:00:20.000Z');
    assert.equal(row.payload_time, row.updated_at);
  }
  db.close();
  console.log(`${name}: counts, response outcomes, compact IDs${name === 'baseline' ? ', and changed writer timestamp' : ''} verified`);
}

make-measurements.py — complete file

import html
import json
from pathlib import Path

root = Path(__file__).parent
lines = ['# Measured requests', '', 'Seconds; excludes the explicit mid-query cancellation probe.', '',
         '| Fixture | Phase | Count | Successful | Deadline aborts | Observed duration |',
         '| --- | --- | ---: | ---: | ---: | --- |']
chart_rows = []
for directory, label in [('baseline', '38,000 activities'), ('scale2', '76,000 activities'), ('scale4', '152,000 activities')]:
    data = json.loads((root / directory / 'results.json').read_text())
    requests = data['results']
    selectors = [('Quiet first process request', lambda x: x['label'] == 'quiet-1'),
                 ('Quiet subsequent', lambda x: x['label'] in ['quiet-2', 'quiet-3', 'quiet-4']),
                 ('Two concurrent, three bursts', lambda x: x['label'].startswith('two-')),
                 ('Four concurrent, first burst', lambda x: x['label'].startswith('four-1-')),
                 ('Four concurrent, second burst', lambda x: x['label'].startswith('four-2-')),
                 ('Two plus twenty metadata changes' if directory == 'baseline' else 'Two plus twenty unchanged-value UPDATE attempts', lambda x: x['label'].startswith('writes-')),
                 ('After mid-query cancellation', lambda x: x['label'] == 'after-cancel')]
    for phase, match in selectors:
        rows = [row for row in requests if match(row)]
        if not rows:
            continue
        times = [row['ms'] / 1000 for row in rows]
        lines.append(f"| {label} | {phase} | {len(rows)} | {sum(r.get('status') == 200 for r in rows)} | {sum(bool(r.get('aborted')) for r in rows)} | {min(times):.3f}–{max(times):.3f} s |")
    quiet = max(row['ms'] for row in requests if row['label'] in ['quiet-2', 'quiet-3', 'quiet-4']) / 1000
    four = max(row['ms'] for row in requests if row['label'].startswith('four-')) / 1000
    chart_rows.append((label, quiet, four))
(root / 'measurements.md').write_text('\n'.join(lines) + '\n', encoding='utf-8', newline='\n')
rows_html = ''
for label, quiet, four in chart_rows:
    rows_html += f'<section><b>{html.escape(label)}</b><div class="track"><div class="serial" style="width:{quiet / 7 * 100:.2f}%"></div><span>{quiet:.2f}s serial</span></div><div class="track"><div class="parallel" style="width:{four / 7 * 100:.2f}%"></div><span>{four:.2f}s with 4 loads</span></div></section>'
page = '''<!doctype html><html><head><meta charset="utf-8"><style>
:root{font-family:var(--font-sans);color:var(--foreground)}body{margin:0}p{font-size:13px;line-height:1.5;color:var(--muted-foreground)}section{margin:18px 0}b{font-size:14px}.track{height:25px;position:relative;margin-top:5px;display:flex;align-items:center;gap:8px}.track:before{content:"";position:absolute;left:85.714%;top:0;bottom:0;border-left:1px dashed var(--warning);z-index:2}.track div{height:14px;flex:none;border-radius:3px}.serial{background:var(--chart-1)}.parallel{background:var(--chart-2)}.track span{font:12px var(--font-mono);white-space:nowrap;position:absolute;left:8px;text-shadow:0 1px 2px var(--background)}.caption{display:flex;justify-content:space-between;font-size:12px;color:var(--muted-foreground)}
</style></head><body><p>Synthetic Lazarus history requests · maximum observed duration<br>Dashed line: 6-second client boundary; scale is 0–7 seconds.</p>''' + rows_html + '''<p>38k and 76k: all successful. At 152k, the first four-load burst timed out; the next succeeded in 5.43–5.48s. Serial bars use subsequent quiet requests. Network and mobile were not exercised.</p></body></html>'''
(root / 'comparison.html').write_text(page, encoding='utf-8', newline='\n')

SHA-256 of the public script files (UTF-8, LF)

7cfd442513dde07786d8b93f3f87c8a047722834671e4053eaa65a2e5cd44354  instrument.mjs
65f8d24933575da3e65e6e42190f85ba15cbda3882db3cfddbe594361df14060  reproduce.mjs
5fab077292d46d6b54ea826d0539e632d883844c08bbf0dd742a57456d6a860d  link-dependencies.mjs
af5e1443639d6a834521a9b86c64c8366f2ec8d185289a208cab38958f8e52bd  summarize.py
6a677557d74fdb0cc3a8934d418f546b5eeec13d0f9fdee6d6af581827eb1893  raw-comparison.mjs
67980f02561645c76cd99cd745e125f65443e0e1ef8fcea531bbc6f116fbd635  verify-artifacts.mjs
65730f36ef6965d41ed0cd891db2ed7469f079e29a11bb1b09afe0ca780ee7e2  make-measurements.py

Reported on the user's behalf by GPT-6.1 Sol (high reasoning), through Codex in T3 Code.

Note

🤖 GPT-6.1 Sol responding on behalf of Alb11747

November 9 recurrence over direct HTTP: Load earlier turns still hit the 6,000 ms deadline on a long thread served by Windows T3 Code 0.1.8-nightly.20261106.4275.fork.1. This connection did not use the managed relay. The screenshot shows the timeout inline with newer turns still visible.

Two matching history GETs used the same cursor and the same deferWork=1&compactIds=1 options. The server traces show:

Request	HTTP span	Timeline index	Transaction cleanup
Canceled	13.75 s; server trace recorded 499	Interrupted at 5.93 s, including 5.56 s in the nested SQLite span	Rollback took another 7.79 s
Later retry	2.74 s; HTTP 200	2.51 s, including 2.05 s in the nested SQLite span	Commit took less than 1 ms

The canceled request acquired a reader in under 0.01 ms. The SQLite span is service-side elapsed time; the 13.75 s HTTP span includes cleanup and is not the query's execution time. This ties the direct-connection timeout to the same history preparation path as the earlier relay report, with the same page succeeding on a later request. It does not establish why the read was slower on the first request.

#20427 tracks the full-thread index rebuild. #24422 proposes reusing that index across pages. That should reduce repeated rebuilds when the cache hits, but the first read on a cache miss still needs to fit within the deadline. The latest published fork build checked (0.1.8-nightly.20261109.4399.fork.2) still calls readTimelineIndex before page selection and uses the same client deadline; I did not reproduce on that build.

Navigation Menu

Repository navigation

[Bug]: Thread snapshot HTTP handlers have no server-side timeout; cold-open herd times out at 6s and floods relay logs with context-canceled warnings #17765

Before submitting

Area

Steps to reproduce

Expected behavior

Actual behavior

Impact

Version or commit

Environment

Logs or stack traces

Workaround

Suggested fix shape (for triage)

Related

Investigation update

Synthetic history follow-up — 2026-11-02

Environment and isolation

Fixture and path exercised

Results

Timing boundaries

Cancellation, separately tested

Limits and next experiment

Source references

Complete measured request matrix

Runnable scripts

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug]: Thread snapshot HTTP handlers have no server-side timeout; cold-open herd times out at 6s and floods relay logs with context-canceled warnings #17765

Description

Before submitting

Area

Steps to reproduce

Expected behavior

Actual behavior

Impact

Version or commit

Environment

Logs or stack traces

Workaround

Suggested fix shape (for triage)

Related

Activity

josephv123 commented on Oct 15, 2026

juliusmarminge commented on Oct 21, 2026

torztomasz commented on Nov 1, 2026

Alb11747 commented on Nov 2, 2026

Investigation update

Alb11747 commented on Nov 2, 2026

Synthetic history follow-up — 2026-11-02

Environment and isolation

Fixture and path exercised

Results

Timing boundaries

Cancellation, separately tested

Limits and next experiment

Source references

Complete measured request matrix

Runnable scripts

Alb11747 commented on Nov 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions