Repository navigation
id: design: terminal rendering ladder — kitty → sixel → iTerm2 → symbols → ASCII, selected by probing #6011
Description
Part of the worlds design program: #5955. Wave-2 follow-up from the prior-art research (Lane B item 38); parent topic issue: #5970. Research source: workspace/scratch/worlds-capabilities-brief-2026-11-04/prior-art-research.md.
The ladder is the design
Modern tooling (chafa, vt, dapple) all implements the same degradation order: Kitty graphics protocol (APC _G, base64 PNG/raw pixels — kitty/WezTerm/ghostty-family) → Sixel (indexed-palette true pixels — xterm/mlterm/foot) → iTerm2 inline images (OSC 1337) → Unicode block/braille/sextant symbols with ANSI color → plain ASCII luminance ramp. Auto-detection probes the terminal and picks the best available; chafa's -f [iterm|kitty|sixels|symbols] with symbols fallback is the reference UX (https://manpages.debian.org/unstable/chafa/chafa.1.en.html ; https://github.com/akirco/vt/blob/HEAD/README.md).
Scope
Distinct from the conversion research under #5955 (media → protocol encoding): this is the client-side selection ladder — probing, choice, and runtime behavior.
- Probing method per rung and its failure modes; tmux passthrough is a recurring breakage and needs explicit handling.
- The symbols/ASCII tiers must be genuinely good, not an afterthought — kitty/sixel support is fragmented across terminals and multiplexers.
- Dead end, recorded: a single "universal" graphics format. The ladder is the design.
Done looks like
A ladder spec (probe sequence, rung obligations, multiplexer rules) the client shells can implement.
Activity
developing-today commentedon Nov 5, 2026
CODE-1677
developing-today commentedon Nov 5, 2026
🤖 Agent note · picked up
developing-today commentedon Nov 5, 2026
🤖 Agent note · done
Triage verdict — LIVE, partially grounded on main, spec routed
Verified fresh 2026-11-05: issue #6011 is OPEN, no hard-stop labels, no other holder. The ladder is not greenfield: (552 lines) already implements terminal capability evidence on main — / flags with response parsers ( ~:271, sixel evidence
:343) feeding a capability struct that already gates behavior on (:59). What the ticket adds on top: the full five-rung ladder as one spec — iTerm2 (OSC 1337) rung, the symbols and plain-ASCII tiers treated as first-class, explicit tmux/multiplexer passthrough rules, and probe failure modes per rung.Dependency map (verified OPEN): parent program issue #5955; parent topic issue #5970 (presentation assets / graceful degradation); capability tiers issue #786. Sibling issue #6010 (semantic cell/tag model) triaged this same shift: its spec execution is task 10548 — the ladder's symbols/ASCII rungs render exactly the tagged cells issue #6010 specifies, so the ladder spec should consume the tag vocabulary v1 rather than inventing its own bottom tiers.
Execution shape (for the follow-up task)
Write the ladder spec per the ticket's Done-looks-like: probe sequence per rung (grounded in the existing evidence model — extend it, don't replace it), rung obligations, multiplexer rules (tmux passthrough named explicitly), and the recorded dead end kept dead (no universal graphics format). The symbols/ASCII tiers get the same design weight as the graphics rungs, per the ticket. Land as a design record on the issue or a docs PR under the standing rules.
41 remaining items
dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
🤖 Agent note · picked up
pkgs/id/src/world_terminal.rsand consume id: design: semantic cell/tag presentation model — one tagged state, many renderers #6010's tag vocabulary.dezren39 commentedon Nov 5, 2026
Proposed terminal-rendering ladder spec v1 (task 10552)
Status: design proposal for client shells; no probe/parser implementation is claimed here. The five rungs are ordered by preferred fidelity: Kitty graphics → Sixel → iTerm2 inline images → Unicode symbols → plain ASCII. The ladder selects among independent capabilities; there is no universal graphics format.
Probe contract and compatibility boundary
Detection runs only in the local client TTY, never in a guest or over the world protocol. Preserve the current
TerminalProfileevidence model and extend it with rung-specific evidence; do not replace the existing fields or infer one capability from another.kitty_keyboardremains orthogonal to visual selection. The currentworld_terminal.rsbehavior is the baseline: one query burst with a single 250 ms deadline, Kitty graphics and Sixel evidence, DEC 2026 synchronized-output evidence, CPR, XTGETTCAPCo, conservative conflict handling, and byte-for-byte replay of unmatched input. Add the iTerm2 query to that same burst—do not add one round trip or timeout per rung. No reply, malformed/partial/late reply, or a reply that conflicts with another complete report is not positive evidence.Selection is top-down, but only on positive evidence:
APC _G…a=q… ST, request id 31). Only the exact matchingi=31;OKreply is positive; a matching protocol error is negative. For the existing interactive/high-frequency renderer, also require the current DEC 2026 synchronized-output evidence. Kitty graphics without that gate does not silently enter the current fast path; continue down the ladder.CSI c); only a complete DA1 reply with parameter4is positive. Keep the existing synchronized-output gate for the current interactive/high-frequency renderer. If Kitty and Sixel are both eligible, prefer Kitty. A positive Sixel flag is not implied by Kitty support or by$TERM.OSC 1337;ReportCellSize ST; accept only a complete documentedReportCellSize=<height>;<width>[;<scale>]response with valid positive dimensions. This proves the OSC 1337 cell-size extension, not that inline-imageFilerendering works—there is no harmless image-specific acknowledgement in this exchange. Therefore treat it as a candidate only when combined with an explicit client profile/verified emulator allowlist for iTerm2 images;TERM_PROGRAMalone is only a hint. Without that policy, fall through rather than claim a false positive or draw a visible canary at startup.CSI 6n); XTGETTCAPCo >= 256enables color but is optional. Use a deliberately small, deterministic, single-column palette (block/braille/sextant symbols) and a visible legend. There is no portable terminal query that proves font coverage, so these are eligibility signals, not a glyph-rendering guarantee; allow an explicit symbols-off override and retain ASCII as the safe fallback. Do not use Kitty keyboard support as a Unicode test.TERM=dumb, no TTY, timeouts, or disabled symbol mode). It must remain a complete representation, not merely a lossy luminance ramp.Keep protocol support separate from rendering policy: DEC 2026 means synchronized output was recognized/toggleable, not that an image protocol exists. Preserve the current
(kitty_graphics || sixel_graphics) && synchronized_outputgate for the existing Tier-1 fast path while the five-rung resolver is introduced. A later full-frame/static-image path without synchronized output needs its own explicit tests; do not silently broaden this contract. Preserve current conservative conflict behavior until transcript tests specify any narrower per-feature conflict handling.Multiplexers, failure behavior, and rung obligations
For tmux,
TERM=screen/tmuxneither proves nor disproves the outer terminal's image support. Only probe/pass through an image protocol when tmux passthrough is explicitly enabled/known; wrap the inner sequence in tmux'sDCS tmux; … STform and double every embedded ESC. tmux documentsallow-passthrough on|all(available since 3.3); do not change the user's tmux configuration. If passthrough is unknown, disabled, filtered, or produces no complete wrapped reply, disable image rungs and continue at symbols/ASCII. Apply the equivalent explicit wrapper rule for any other multiplexer; never assume a generic DCS tunnel.A missing/unsupported response, unsupported mode, invalid dimensions, or timeout falls through to the next eligible rung. Malformed, partial, unmatched, or interleaved user bytes remain input/replay data, never capability evidence. Conflicting complete evidence takes the current conservative fallback. If a selected renderer fails during the session, disable that rung for the session, redraw the full frame at the next rung, and preserve world state/input; do not repeatedly probe on every frame. Re-probe only for a new TTY/reconnect.
All five renderers consume the same authorized tagged-state projection from the #6010 spec: cell
kind, primary/additional semantic tags, entitled identity, and affirmative state flags. The renderer changes presentation fidelity, not world semantics. Image protocols transport resolved pictures; they do not define tag meaning. Every rung needs a total deterministic mapping for its supported tags, and an unknown tag must remain inspectable through safe text rather than disappear. Symbols and ASCII use the #6010 vocabulary and retain details via a legend/description when the glyph cannot encode them (e.g. a blue-caped, sparkling player must not become an unexplainedC). The lowest rung's oracle is equality of the normalized recipient-visible semantics before rendering and after decoding the glyphs plus legend.Verification oracle
Extend transcript tests with positive/negative Kitty, DA1 Sixel, OSC 1337 geometry, combined replies and priority, and cases for no reply, malformed/partial/late replies, contradictory evidence, interleaved input replay, and tmux-wrapped versus unwrapped replies. The decisive design oracle is a shared #6010 tagged-cell fixture rendered through every rung:
kind, entitled identity, tags, and flags survive in pixels/glyphs plus legend; ASCII-only output cannot rely on color, emoji, or Unicode. The current parser tests validate only the existing evidence model, not the new iTerm2 or symbol probes.Source anchors: current probe/evidence model; #6010 tagged-state spec; Kitty graphics query; iTerm2 ReportCellSize; tmux passthrough rules.
🤖 Agent note · done
cargo test --manifest-path pkgs/id/Cargo.toml --lib world_terminal::tests— pass (7 passed, 0 failed); parser source inspected atorigin/main.