SSE Stream Bug: Out-of-Order thinking_delta via LiteLLM → AWS Bedrock #3596

Reported with help from Claude Haiku 4.5 and OpenCode v0.15.29

Issue Summary

Proxying Claude Haiku 4.5 through LiteLLM → AWS Bedrock Converse API occasionally delivers thinking_delta blocks out-of-order, causing OpenCode stream processor crash.

• Problem: thinking_delta arrives BEFORE message_start (intermittent, unpredictable)
• Impact: state machine violation → stream crash → incomplete response
• Frequency: rare, unknown cause
• NOT: Anthropic or Bedrock official API issue

Root Cause Analysis

Request Path

OpenCode → LiteLLM → AWS Bedrock Converse API
  ↓
LiteLLM reconstructs Bedrock response as Anthropic SSE
  ↓
Out-of-order blocks, missing cache tokens (evidence: cache_read_input_tokens = 0)
  ↓
OpenCode stream processor crashes

Evidence LiteLLM Is Reconstructing

• Cache tokens all zero: indicates response recomposition, not direct forwarding
• Block order violation: Bedrock typically maintains order for direct clients
• Specific to LiteLLM proxy: doesn't occur with direct API calls
• Conclusion: LiteLLM's stream handler buffers → reconstructs → emits SSE with modified order

Suspected Mechanism

LiteLLM's stream() method:

1. Buffers chunks from Bedrock
2. Reconstructs Anthropic-compatible structure
3. Emits SSE with reordered blocks
4. Loses metadata (cache tokens)

Failure Scenario

Observed sequence (malformed):

1. content_block_delta (index: 1, type: thinking_delta) ❌ BEFORE message_start
2. message_start
3. content_block_start (index: 0, type: text)
4. content_block_delta (index: 0, type: text_delta) × N
5. ...

Expected (correct):

1. message_start
2. content_block_start (index: 0, type: text)
3. content_block_delta (index: 0, type: text_delta) × N
4. ...

OpenCode processor behavior:

• Stream handler expects: text-start → text-delta+ → text-end
• Receives: mixed text-delta + reasoning-delta interleaved
• State machine violation → updatePart() fails → crash

Impact

Proposed Solutions

Option 1 (Simple): Skip unhandled delta types (e.g., thinking_delta) in stream processor, continue processing valid blocks → solves immediately.

Option 2 (Robust): Initialize missing start events on-demand + wrap updatePart in try-catch → handles multiple failure modes.

Option 3 (Recommended): Combine both → prevent issues upstream + graceful degradation downstream.

Context

• Not Anthropic official API: confirmed different behavior with direct API
• Not Bedrock issue: Converse API maintains order for direct clients
• LiteLLM proxy layer bug: response reconstruction doesn't preserve ordering
• Intermittent: cause unknown, no reliable reproduction steps
• Components affected: stream processor
• Related: cache token metadata loss confirms response recomposition

Repro cases

event: message_start
data: {"type":"message_start","message":{"id":"msg_test","type":"message","role":"assistant","content":[],"model":"claude-haiku-4.5","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":0,"output_tokens":0}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"The architecture"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" has three layers:"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\n1. Parser layer - handles input"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n2. Type guard layer - validates types"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n3. Domain model layer - business logic"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\nThis pattern is useful for UI builders"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" and query optimization."}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"tooluse_test123","name":"TodoWrite","input":{}}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}}

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"input_tokens":1000,"output_tokens":300}}

event: message_stop
data: {"type":"message_stop"}