Repository navigation
[FEATURE] Recursive Language Model (RLM) Context Management - Context as External Environment #11829
Description
TL;DR
Instead of managing what fits in the context window (compaction, sliding window), treat context as an external environment the model queries programmatically. This is the RLM paradigm from MIT's arXiv:2512.24601 paper - and it's production-ready in 2026.
Builds upon: #4659 - Sliding window context management by @rickross
Research: Recursive Language Models (arXiv:2512.24601)
Inspiration: Signal Zero symbolic cognition engine
The Problem
Current compaction approaches are lossy:
[Full Context] → OVERFLOW → [AI Summary] + [Recent Work]
The model loses:
- Nuanced reasoning chains
- Architectural decisions
- Failed approaches (negative knowledge)
- Cross-reference relationships
Even the excellent sliding window approach from #4659 still manages what fits in the window. It's smarter about what to keep, but the paradigm is the same.
The RLM Paradigm Shift
From the MIT paper:
"Long prompts should NOT be fed into the neural network directly but should instead be treated as part of the EXTERNAL ENVIRONMENT that the LLM can symbolically interact with."
Current paradigm:
Context → [Stuff into Window] → Model
RLM paradigm:
Context → [External Storage] ← Model queries via code → [Relevant Snippets] → Model
Performance Evidence
| Task | Base LLM | Compaction | RLM |
|---|---|---|---|
| BrowseComp+ (10M tokens) | 0% | ~20% | 91% |
| DeepDive (Long Context) | 45% | 52% | 78% |
| Oolong (Retrieval) | 38% | 48% | 71% |
RLM scales to 10M+ tokens - two orders of magnitude beyond context windows.
How It Would Work in OpenCode
1. Context as External Environment
Instead of packing messages into the context window:
// Current approach
const messages = session.messages.slice(-N);
await model.generate(messages);RLM approach:
// Context stored externally
const contextStore = new ContextEnvironment(session.messages);
// Model writes code to query what it needs
const queryCode = await model.generateCode(`
# Find relevant context about authentication
relevant = context.search("authentication", limit=10)
recent = context.slice(-20) # Last 20 messages
return merge(relevant, recent)
`);
// Execute query, inject results
const snippets = await contextStore.execute(queryCode);
await model.generate(systemPrompt + snippets);2. Available Query Operations
# The model has access to:
context.search(query: str, limit: int) -> List[Message]
context.slice(start: int, end: int) -> List[Message]
context.filter(predicate: Callable) -> List[Message]
context.regex_search(pattern: str) -> List[Message]
context.get_by_id(id: str) -> Message
# Sub-LLM delegation for complex analysis
sub_llm(prompt: str) -> str
sub_llm_batch(prompts: List[str]) -> List[str] # Parallel3. Sub-Agent Decomposition
For massive contexts, the model spawns sub-agents:
# Analyze 100k tokens across 10 parallel sub-agents
chunks = context.chunk(size=10000)
analyses = sub_llm_batch([
f"Summarize security concerns in:\n{chunk}"
for chunk in chunks
])
synthesis = sub_llm(f"Combine findings:\n{analyses}")Integration with #4659 Concepts
This complements rather than replaces @rickross's sliding window approach:
| #4659 Concept | RLM Integration |
|---|---|
| Inception Messages | Always-loaded symbols in the context store |
| Chess-Clock Time | Weighting factor for search relevance |
| Priority Levels | Cache priority in context store |
| Heuristic Pruning | Background optimization of context store |
The ACM tools from #4659 (acm_preserve, acm_prune, etc.) become metadata for the context store, not filtering logic.
Symbolic Compression Layer (From Signal Zero)
For extreme efficiency, frequently-accessed context can be compressed to symbolic representations:
// Instead of 500 tokens of architectural context
const fullContext = "This system uses event sourcing...";
// Compressed to ~10 tokens
const symbol = {
id: "ARCH-EVENT-SOURCING",
triad: ["events", "broadcast", "persist"],
expand: () => loadFullContext("ARCH-EVENT-SOURCING")
};The model references symbols in queries, expands only when needed.
Implementation Phases
Phase 1: Context Store Foundation
ContextEnvironmentclass for external message storage- Basic search/filter operations
- Semantic search via embeddings
- Integration with existing session storage
Phase 2: Code-Based Query System
- Sandboxed REPL for context queries
- Query result injection into model context
- Safety guards for code execution
Phase 3: Sub-Agent Delegation
- Sub-LLM spawning for context decomposition
- Parallel execution with
sub_llm_batch - Token budget management
Phase 4: Symbolic Compression
- Triad generation for frequent context
- On-demand expansion
- Integration with inception messages
Phase 5: Hybrid Mode
- Combine sliding window + RLM
- Graceful fallback for simpler models
- User controls for query depth
Questions for Maintainers
-
Paradigm alignment: Does treating context as external environment align with OpenCode's direction?
-
Hybrid approach: Should RLM complement compaction, or eventually replace it?
-
Plugin vs core: Could this be a plugin, or does it require core changes?
-
Sub-agents: Is spawning sub-LLMs acceptable for cost/latency tradeoffs?
-
Model requirements: Should this be limited to models with strong code generation (Claude, GPT-4+)?
Why Now?
RLM is being called "the most exciting Agentic Paradigm of 2026" (VentureBeat, Jan 2026). The MIT paper has implementations on GitHub (github.com/alexzhang13/rlm). This isn't future speculation - it's production-ready research.
Combined with @rickross's battle-tested ACM tools from #4659, OpenCode could have the most sophisticated context management in any coding agent.
References
- Issue [FEATURE] Sliding window context management for long-running sessions #4659: Sliding window context management - Tactical improvements (inception, chess-clock, priorities)
- arXiv:2512.24601: Recursive Language Models - The paradigm shift paper
- Signal Zero: signal-zero.ai - Symbolic cognition with 6+ month session continuity
- Prime Intellect Blog: RLM Overview - Production implementation notes
This proposal is research-backed discussion, not a PR. Looking for maintainer input on whether this direction aligns with OpenCode's vision before implementation work begins.
Activity
github-actions commentedon Feb 2, 2026
This issue might be a duplicate of existing issues. Please check:
Feel free to ignore if none of these address your specific case.
klietus commentedon May 15, 2026
Holy shit. I'm the builder of Signal Zero. Didn't know anyone was watching. 🤣
There is a lot to this. Including Hebbian learning, EMA decay to side step state explosion, graph topology refactoring by the back end, context window layering by stability.
I encourage you to dive the code base if you need examples. Happy to consult on why it's useful.
https://github.com/klietus/SignalZero-Desktop
klietus commentedon May 15, 2026
There are a few things that this feature request is missing for maximum precision and efficiency. The symbolic format uses Peircean triads of emojis to leverage semiotic weight in latent training space. They are more precise than natural language.
Also I added semantic overlays (activation conditions) that actually get indexed in lancedb to catch the query expansion of the user prompt. Those are what tell the LLM when to load the symbolic format. Both can be generated by the LLM on symbol creation.
If you link together the nodes into a knowledge graph and give it an adaptive search those ids act as lazy loading hooks for the LLM, then you just return the IDs and triads to give the LLM minimal context to decide when to load the full payload.
Note that the triadic signatures are a reduced dictionary of causal flow of the concept. You can use cosine nearness of those representations to auto link together similar concepts in the graph.
jphein commentedon May 16, 2026
Operator note from an adjacent project — we've been running RLM (via
alexzhang13/rlm, the paper's own reference implementation) as a memory-system orchestrator in production for ~3 weeks, on a 274K-drawer palace + 200-question Cat-9a-shaped retrieval eval. Adds three operational caveats to the "RLM is production-ready" framing that aren't in the published numbers:1. Base model choice matters more than RLM-the-framework. Same RLM wrapper, same task, same retrieval substrate —
gemma4:e4blands at 0.417 mean recall;qwen3.5:4blands at 0.717 (+30pp at fixed 4B params). The published Tau2 tool-use benchmark predicts this almost exactly — qwen3.5 leads gemma4 by 37.7 Tau2 points. So when OpenCode adds RLM, the base model's tool-use training is the dominant variable, not the RLM scaffold. The published "91% BrowseComp+ with RLM" numbers presumably use a capable base; with a 4B local model the floor is much lower.2. The RLM "model writes code to query what it needs" assumes the model will write the code. On our 30-question diagnostic, gemma4 chose to answer from prior knowledge on 18/30 questions (60% zero-call rate) — never invoked the search tool at all. qwen3.5 zero-called on only 4/30 (13%). RLM's elegance depends on the model deciding to invoke; lower-discipline models defeat the paradigm. Two-axis sub-test:
The 30pp recall gap decomposes ~50/50 into "does the model invoke?" and "what does it do with the result?"
3. Prompt-engineering controls partially close the gap. We prepended a
MANDATORY RETRIEVAL CONSTRAINTdirective to RLM's standard system prompt (system-prompt-only, no library change). gemma4 lifted from 0.417 → 0.567 at n=5 (+15pp) and → 0.724 at n=20 (+30.7pp) on the same task. The fix touches one constructor kwarg (example) and surfaces asinvocation_mode="forced"in our wrapper. Caveat: at n=20 the same intervention regressed qwen3.5 (0.883 at n=5 → 0.850 at n=20) — forced retrieval interacts non-additively with retrieval breadth. Recommendation for the OpenCode side: expose a system-prompt-augmentation knob alongside the RLM tool surface, not just the tool surface; and sweepkrather than fixing it. Low-discipline base models become usable when invocation isn't optional; high-discipline models can be hurt by over-forcing.4. The substrate can already beat the orchestrator. Raw
mempalace-daemon(no LLM orchestration, just hybrid postgres+pgvector+AGE retrieval) hits 0.733 recall on the same corpus where gemma4-RLM lands at 0.417. The orchestrator was actively dropping signal the substrate provided for free. RLM's value isn't automatic — it requires the base model to use retrieved content well. Worth measuring "RLM lift vs raw retrieval" as a separate axis in OpenCode's benchmarks; we expect non-trivial regressions on small models.Full per-model breakdown and the forced-invocation comparison live on the SME tracking issue (
M0nkeyFl0wer/multipass-structural-memory-eval#3) — Step 2-forced (4 conditions × n=5/n=20) just landed with the full table and decomposition. Cross-linking here in case the empirical lens helps shape OpenCode's RLM rollout.The published RLM numbers are real, but they're a ceiling that depends on (a) base-model tool-use quality and (b) the model choosing to invoke. Both are configurable; both deserve first-class surfaces in any RLM integration. 🫏
chindris-mihai-alexandru commentedon Jun 14, 2026
@jphein @klietus really appreciate your comments!
Tbh, I've recently moved away from using OpenCode as my daily driver and now primarily use AdaL CLI from Sylph AI. This change actually led me to build a tool called adal-compress, which is a local HTTP proxy that compresses token payloads at the wire level before they reach the model. I've made it available on GitHub at https://github.com/chindris-mihai-alexandru/adal-cli/tree/main/adal-compress.
One of the benefits of this approach is that it doesn't require any specific invocation discipline, it just applies deterministic rule-based compression to every request. I have to say that I kept running into the issue of the substrate beating the orchestrator, but by using rule-based compression at the proxy layer, I've been able to sidestep this problem entirely, and in practice, I've seen savings of 50-93% on exploration-heavy sessions.
The work being done on building a queryable memory that the model navigates is a different and harder problem than what I've been working on. While I'm focusing on compressing the data that gets sent, this other approach is building a complex system with multiple components, and I'll def have a look over the SignalZero-Desktop codebase to learn more about it.
klietus commentedon Jun 14, 2026
Try an emoji transformation for true token compression. It's absurd. 😁
meotisR commentedon Jun 22, 2026
👀
TomLucidor commentedon Jul 26, 2026
As a bit of heads up please check the following harness, some of the patterns might be useful somehow https://github.com/itayinbarr/little-coder
github-actions commentedon Sep 25, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.