provider: big-pickle leaks chain-of-thought into visible text when reasoning contains the literal close-think tag #55268

Activity

1. hollychan05-ctrl commented on Oct 14, 2026

   

   hollychan05-ctrl

   on Oct 14, 2026

   Author

   More actions

   Copy/paste note for the reproduction: the closing tag is exactly the 8 bytes hex 3c 2f 74 68 69 6e 6b 3e (less-than, slash, t h i n k, greater-than).

   The tag renderings elsewhere in this report are visually identical but may carry an invisible zero-width character depending on how they were copied. If the reproduction does not trigger, retype the tag from those bytes (in JS: "\u003c/think\u003e").
2. hollychan05-ctrl commented on Oct 14, 2026

   

   hollychan05-ctrl

   on Oct 14, 2026

   Author

   More actions

   Update (opencode 2.0.29): the context-side workaround is not sufficient in tag-heavy sessions — the model reconstructs the tag and still splices

   New evidence from a session where the client-side defuse was active the whole time (context defused {"tags":N} fired on every turn, i.e. every literal tag in the outgoing messages was neutralised with an inserted U+200B).

   Session ses_ec5e33619ffeyWCNsuzx9N6pKp, model {"id":"big-pickle","providerID":"opencode"}, opencode 2.0.29.

   From opencode.db → session_message (type='assistant' → data.content):

   seq	time (UTC)	reasoning len	text len	observation
   165	11:15:29	1959	4973	text starts mid-reasoning: ` inside its reasoning; the provider treats it as end-of-reasoning and routes the *rest of…
   180	11:15:51	411	6941	reasoning ends …模型在 thinking 裡吐出字面 ` and text starts `；供應商把之後的 thinking 切到 content 通道…
   192	11:17:02	806	6940	reasoning ends mid-word; text continues the same chain-of-thought

   In 180/192 the leaked fragment is prepended to the legitimate answer inside the same text part (the real answer is intact after the fragment).

   Why the workaround no longer holds here: the trigger is no longer the incoming context but the model's own generated reasoning. Because the session itself discusses the tag (its plugin source, logs and README are in context), the model reconstructs the canonical bytes (3c 2f 74 68 69 6e 6b 3e) in its reasoning and emits them, so defusing the outgoing messages cannot prevent it. This confirms that the context workaround is a mitigation only and raises the priority of the provider-side fix: a model-emitted close tag must not terminate the reasoning channel / switch the delta channel.

   Also note the stream-side net is not a substitute: when active it drops leading characters of otherwise legitimate answers (rewrite {"event":"cleaned","dropped":23} observed).

   Environment: opencode 2.0.29; Windows 11 (10.0.26200, x64); shell cmd.exe; provider/model opencode/big-pickle.
3. hollychan05-ctrl commented on Oct 14, 2026

   

   hollychan05-ctrl

   on Oct 14, 2026

   Author

   More actions

   Update: wire-level capture of the second form (missing opening tag) — the client cannot recover from it

   Same reporter, same machine. Since my original report I captured the raw provider SSE while a session was actively leaking, and it shows a second failure mode that is worse than the tag-splice one because nothing is left for the client to detect.

   Environment

   • opencode 2.0.29 (was 2.0.28 in the original report; the leak survived the upgrade)
   • Windows 11 10.0.26200 x64, cmd.exe
   • Model opencode/big-pickle (default variant), agent build
   • Capture: an http.response hook on responses coming from providerID: "opencode" that tees the SSE body and logs the raw chunks (no rewriting)

   Wire evidence (live leaking session, 2026-10-14T12:14Z)

   Self-contained excerpt — this is the complete response of one assistant step (generated id style router-…, 6 data: lines):

   {"delta":{"role":"assistant","content":"","reasoning_content":null}}
   {"delta":{"role":null,"content":"</think\u200b>","reasoning_content":null,"tool_calls":null}}
   {"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"…","function":{"name":"…","arguments":"…"}}]}}
   {"delta":{"reasoning_content":null},"finish_reason":"tool_calls"}
   {"choices":[],"usage":{"prompt_tokens":43656,"completion_tokens":80,"total_tokens":43736,"prompt_tokens_details":{"cached_tokens":43008}}}
   

   Nine consecutive captured responses from that session, all identical in shape:

   time (Z)	data lines	bytes on content	bytes on reasoning_content	opening tag in content	closing tag in content
   12:14:03	6	8	0	no	yes
   12:14:11	123	≥1200 (probe cap)	0	no	no
   12:14:14	46	427	0	no	yes
   12:14:17	6	8	0	no	yes
   12:14:23	199	≥1200	0	no	no
   12:14:49	474	≥1200	0	no	yes
   12:14:50	6	8	0	no	yes

   Points that matter:

   1. delta.reasoning_content is present as a key in every chunk but is always null. Chain-of-thought is streamed on delta.content.
   2. The opening tag never appears in content. The closing tag does (and often arrives as its own isolated chunk — 8 bytes — immediately before a tool_calls chunk).
   3. usage.reasoning_tokens / opencode's tokens.reasoning is 0 for these messages, so the client-side token accounting agrees with the provider: as far as the API is concerned, no reasoning was produced.

   Same probe, same model, ~19h earlier: reasoning_content was populated

   On 2026-10-13T17:28–17:44Z the same hook captured 51 responses from opencode/big-pickle on this machine; 24 of them carried reasoning text:

   17:29:25 | dataLines=112  | content=173 bytes | reasoning=900 bytes (probe cap)
   17:29:59 | dataLines=1376 | content=900       | reasoning=900
   17:30:21 | dataLines=2217 | content=900       | reasoning=900
   17:43:50 | dataLines=1129 | content=502       | reasoning=900
   

   So reasoning_content can and did carry the CoT for this exact model through this exact gateway. Today it never does. That asymmetry is on the provider/gateway/generator side — by the time the response reaches opencode there is nothing to separate.

   Why this cannot be fixed in the client

   opencode splits reasoning from visible text purely on the API fields (reasoning_content / reasoning / reasoning_details). There is no tag-based fallback: the byte sequence </think does not occur anywhere in opencode.exe. When the provider reports no reasoning field and no reasoning tokens, opencode stores the whole string as a text part and renders it — there is no local signal left to key on.

   DB-level shape of the two forms

   Both were observed in the same install (session rows in session_message, data JSON):

   form	reasoning part	tokens.reasoning	visible text part
   A — tag splice (original report)	present, ends mid-sentence	> 0 (e.g. 141, 186, 653)	CoT remainder + literal </think\u200b> + answer
   B — missing opening tag (this comment)	absent	0	whole CoT + trailing </think\u200b>

   Form B now dominates:

   • Session ses_ec5fafc6… ("performance triage"): 94 of 99 assistant turns have no reasoning part; the leak is present from the second assistant turn onward and still present at the 99th, ~65 minutes later.
   • Session ses_ec5e336… (this report's own triage session): the first 45 assistant turns all had a reasoning part; then from 11:56:43Z 15+ consecutive turns have none, with individual text parts of 8k–22k characters of raw CoT.

   It is self-reinforcing

   Once the stored history contains assistant turns whose text is "CoT … </think\u200b>", every later turn reproduces that template: the model stops emitting the opening tag, so the provider never opens a reasoning channel, so the next turn is stored the same way. Both sessions show exactly this: clean first turns → then permanently form B, surviving session restarts (and a 2.0.28 → 2.0.29 upgrade). It is not caused by plugins — the split happens at the wire, upstream of any context/http.response hook, and the same capture shows the provider's bytes before the client persists anything.

   Suggested fix direction

   • Provider/gateway: do not gate the reasoning channel on an opening tag the model may omit, and do not split a reasoning stream at a tag occurrence that the model can legitimately write inside its reasoning. If the model ends its reasoning with </think…>, the text before it is still reasoning.
   • opencode (defensive): if a whole assistant message arrives with no reasoning_content and reasoning_tokens == 0, but content contains a closing think tag (tolerant of the Unicode zero-width suffix characters, as plugins already have to be), treat the prefix as reasoning instead of visible text. That would at least stop form B from being persisted and re-fed as visible history.

   Related reports: #54586 ("model reasoning always visible, because starttag is missing", closed as not planned), #54599, #54886.
4. hollychan05-ctrl commented on Oct 14, 2026

   

   hollychan05-ctrl

   on Oct 14, 2026

   Author

   More actions

   Addendum: the splice point of form A is an inline-code quotation of the tag

   Reconstructed from the persisted parts (session_message.data.content[]). Form A — a reasoning part that stops mid-sentence with the remainder visible — does not cut at an arbitrary byte. It cuts exactly where the model wrote the tag inside inline code, i.e. `</think…>`.

   Reasoning tail vs. visible-text head, verbatim (several sessions, 2.0.28 and 2.0.29):

   R: ... the model emits the literal 8-char `
   T: ` inside its reasoning; the provider ...
   R: ... 根因: 模型在 thinking 裡吐出字面 `
   T: `；供應商把之後的 thinking 切到 content 通道 ...
   R: ... everything until a `
   T: ` is discarded. Since leak form is ...
   R: ... reasoning continued: "...the literal `
   T: ` tag — yes! In my answer I wrote ...
   

   So the model's stream is

   <CoT> … the literal 8-char `</think​>` inside its reasoning … </CoT><answer>
                             ^^^^^^^^^^^ the byte sequence the provider matches on
   

   and the provider behaves as if reasoning ended at that offset:

   1. the reasoning part keeps everything up to the tag (so it ends right after the opening backtick, mid-sentence);
   2. the 8 tag bytes are consumed — they do not appear on content;
   3. the remainder is routed to the visible channel, which therefore starts with the closing backtick of the quotation.

   That last point is a useful contrast with form B (the reasoning_content: null form from my previous comment): there the tag is not consumed — my wire capture literally shows {"delta":{"content":"</think\u200b>"}}, i.e. the provider forwards those bytes verbatim when it is not in reasoning state. So the same byte sequence is treated as a control token in one state and as ordinary text in the other.

   How often

   Per session, counting assistant messages that have a reasoning part, how many end mid-sentence, and how many of those end at a backtick:

   session	msgs with reasoning	cut mid-sentence	of those, cut at a backtick
   session A	51	5	4
   session B	24	2	2
   session C	7	2	1
   session D	15	1	1
   session E	5	1	1

   (The same sessions have the tag sitting in visible text parts in 14–66 messages each.)

   Implication

   The reasoning channel is terminated by the model's own output, and the trigger is any mention of the tag — a bug report like this one is the worst case, because the tag is then quoted in inline code almost by construction. Nothing on the client side can prevent it: by the time the response is persisted, the reasoning part is simply truncated and the remainder is ordinary text.
5. chfabbro commented on Nov 7, 2026

   

   chfabbro

   on Nov 7, 2026

   More actions

   Same class of failure on the OpenRouter/DeepSeek lane, with an opening tag inside the reasoning as the trigger. Raw SSE and repro here: #54861 (comment)