mcpx docs: an agent-markdown twin of every page, and the bot home page carries the install prompt #1947

Relocated from dezren39/darwin-nix#120 on 2026-10-28 (placement audit): mcpx's single source lives in developing-today/code (pkgs/mcpx); re-home tracked at #309. Original body below; issue references inside it are historical (dezren39/nix numbering).

Parent: #90

Summary

Every page of the docs site gets a machine-readable twin written for agents. On top of that, the site publishes llms.txt and llms-full.txt. The agent twin of the home page carries a one-shot install prompt: a procedure an agent can execute to install mcpx, verify it, and wire it into the harness it is running in. The same agent docs should also ship inside the binary, so an offline agent gets docs that match its version.

Original notes

• documentation
  • make a book using whatever is the most common and simple modern choice
    • specialized agent markdown all pages
      • home page has install prompt built into the bot version

Current state

• AGENTS.md (109 lines) is already agent markdown: the loop (ls / search / types / run|exec), scripts, profiles, the script contract, results, stateful servers, and a do-not list. It is only a file in the source tree:

  • package.nix installs the binary, the man page and completions, not AGENTS.md;
  • the only go:embed is internal/defaults/defaults.json.

  An agent that has mcpx but not the repository never sees it.

• mcpx man renders roff only. Its one flag is --install DIR (internal/cli/root.go:257). mcpx help [command] renders the same Commands() table (internal/cli/manual.go) as text. Neither emits markdown.

• Plugin install today is copying files (plugin/opencode/README.md "Install": cp -R mcpx-session.ts mcpx ~/.config/opencode/plugin/, skills separately). There is no package. Since chore(deps): bump build/buck/preludes/buck2-prelude from d956689 to 880be56 #53 (closing Bump rustls-webpki from 0.100.1 to 0.100.2 in /sources/pavex/examples/realworld #44) the plugin works without the binary: it reaches the daemon over the unix socket and spawns mcpx --json status only as the last-resort rung 4 of its discovery ladder (plugin/opencode/README.md:99-138). An install prompt has to fetch these files from somewhere (amd host fails at evaluation: no outputHashes for p2panda-core-0.7.1 #158).

• mcpx doctor [-v] already "diagnoses the installation" and says what to do about each failed check (manual.go:261). It is the natural verification step, alongside mcpx ls.

• No site exists yet (chore(deps): update rust crate actix-files to v0.6.10 - autoclosed #117, fix(deps): update github.com/charmbracelet/ssh digest to ebfa259 - autoclosed #118).

The conventions (llms.txt v2)

Fetched from llmstxt.org (v2, modified 2026-08-10):

• /llms.txt is markdown in a fixed order:

  1. an H1 with the project name (the only required part);
  2. a blockquote summary;
  3. optional non-heading prose;
  4. H2 sections that are "file lists" of [name](url): notes.

  An ## Optional section by convention holds links an agent can skip. The file may sit at any path and covers the pages under it, e.g. /docs/llms.txt.

• A clean markdown version of each page at the same URL, with .md appended (page.html.md) or replacing the extension (page.md). URLs without a file name use index.html.md / index.md.

• Discovery: <link rel="alternate" type="text/markdown" href="...md"> in each page, and rel="describedby" pointing at the llms.txt that covers the page. These can also be sent as an HTTP Link: header, but GitHub Pages cannot set headers, so we use HTML <link> elements only.

• llms-full.txt is not in the spec. It is a widely used convention: every page concatenated, used by Starlight, Docusaurus and Mintlify plugins.

• No content negotiation. GitHub Pages is static: it cannot answer Accept: text/markdown with the markdown twin at the same URL, and it cannot set Link: headers. Agents have to be pointed at .md URLs explicitly (in llms.txt, the <link> element and the home page). Say so on the site, so nobody expects negotiation to work.

Caveat: Zensical's own analysis (ZAP 009) notes limited evidence that vendors' agents consume llms.txt. The per-page .md twin and the embedded docs are the parts that pay off regardless. llms.txt is cheap once the twin exists.

What "specialized" means

A raw copy of the markdown source is not enough. An agent page should:

• Drop decoration: the video, copy buttons, the rotating prompt, <details> wrappers (expanded instead), badges, theme HTML.
• Inline what the HTML page links or includes: the generated reference (🎨 Palette: Improve accessibility of editor search panel #144), i.e. settings, flags and env vars, command usage, as plain markdown tables or text, with {{#include}} and shortcodes already resolved. Zensical makes the same point: markdown must be an output of the pipeline, not a copy of the source.
• Carry agent instructions:
  • a short header (what this page is for, what to run next, the one command that answers the page's question: mcpx help <cmd>, mcpx config --schema, mcpx types <ns>);
  • version-matched: "these docs describe mcpx vX.Y. If mcpx --version differs, prefer mcpx help."
• Be sourced from AGENTS.md where it applies. It already is the agent-facing instruction file, so the site's agent pages for the loop, scripts and results should be derived from it or include it, not rewritten. There should be one agent-facing text, not two that drift.
• Keep the version history (🎨 Palette: Add ARIA labels and live regions to search panel #150) flattened but present. Agents need "since vX" more than humans do.

The bot home page: the install prompt

/index.md and the top of /llms.txt carry a procedure an agent can execute end to end, written as numbered imperative steps with the exact commands and the checks between them:

1. Detect OS and architecture, the available package managers (nix, brew, go, ...) and the harness it is running in (opencode, claude code, codex, ...).
2. Pick the channel from the list in amd host fails at evaluation: no outputHashes for p2panda-core-0.7.1 #158: nix profile / flake, Homebrew, go install, release binary, curl <short> | sh. Prefer one that installs per-user and needs no root. Say which channel was chosen and why.
3. Install, then verify: mcpx --version, mcpx doctor, and mcpx ls, which must list namespaces or say clearly that none are configured.
4. Wire the harness plugin for the detected harness: opencode via the plugin files (today a copy; a package once amd host fails at evaluation: no outputHashes for p2panda-core-0.7.1 #158 exists), or others via Update Rust crate anyhow to 1.0.75 #48's adapters or mcpx as an MCP server. Then verify from inside the harness.
5. Optionally move the user's existing MCP servers behind mcpx. After the agent shows the diff and gets confirmation, it comments out (never deletes) the user's existing MCP server entries in the harness config, so that the servers are reached through mcpx instead of loaded directly. This echoes the home page prompts in 🎨 Palette: Add aria-labels to buttons and inputs #155. Whether mcpx should import them first (mcpx init or registry add) is an open question.
6. Report what was installed, where, and how to undo it.

The human home page (#155) shows the same prompt behind a copy button. The bot page is the prompt, with nothing around it. Keep a single source for the text so the two cannot drift.

Shipping the agent docs in the binary

An agent with mcpx on PATH but no network, or with a different version than the site, should get docs that match the binary it has.

Prefer a format flag on an existing command over a new command:

• mcpx help --format markdown (or mcpx man --format markdown|roff) prints the agent version of the command reference, generated from Commands() like the man page.
• mcpx help agents (or a topic argument) prints the embedded AGENTS.md.

Coordinate with #144: the markdown emitter that feeds the site's reference pages should be the same one behind this flag, so the site's agent pages and the binary's output are byte-identical for the same version. Embedding AGENTS.md costs a few KB, and go:embed is already in use.

Approaches: how the twin is produced

Approach A (least recommended): generator plugin only

Examples: starlight-llms-txt, docusaurus-plugin-llms, mdbook-llms-txt-tools, or Hugo's built-in markdown output format with .RawContent.

• It is fast to turn on.
• But these plugins emit the source (or lightly cleaned HTML-to-markdown): shortcodes and includes are raw or dropped, decoration leaks in, there is no agent header, and the output shape changes with the plugin version.
• It ties the twin to the generator chosen in chore(deps): update rust crate actix-files to v0.6.10 - autoclosed #117, and cannot produce the in-binary docs.

Approach B (middle): a build step emits .md + llms.txt from the same source

• A small program in this repo reads the book after preprocessing and writes, per page:
  • <page>.md, with decoration stripped, includes resolved and an agent header prepended;
  • llms.txt (ordered like the site navigation, one ## per top-level section, with a one-line note per page taken from its lede);
  • llms-full.txt.
• The source can be an mdBook preprocessor over the JSON book, as in fluree/db#1340, or a Hugo template.
• It adds <link rel="alternate" type="text/markdown"> to every HTML page via the theme.
• The result is deterministic, generator-light and checkable in CI. It still only derives the twin mechanically. Pages that need a different shape for agents (the home page, the install prompt) have no place to live.

Approach C (recommended): B, plus hand-written agent sources where they add value, plus the same emitters in the binary

• The twin is derived by default, as in B, for every page.
• A page can carry an agent-only override or supplement: a sibling page.agent.md, or a fenced agent block the HTML renderer hides. The bot home page / install prompt and the loop from AGENTS.md live there.
• AGENTS.md becomes a source of the site rather than a separate document.
• The reference parts use the same Go emitters as mcpx help --format markdown (🎨 Palette: Improve accessibility of editor search panel #144 / ⚡ Bolt: Optimize cursor color generation and position grouping #145), so the site's twin and the binary agree for a given version.
• CI checks that every HTML page has a twin, every twin is linked from llms.txt, and the twin is current. Docs checks run in CI; see the testing trackers under Bump github/codeql-action from 2 to 3 #91 (CI pipeline Bump golang.org/x/crypto from 0.16.0 to 0.17.0 in /sources/launcher #95, integration/environments Bump golang.org/x/crypto from 0.14.0 to 0.17.0 in /sources/chat #97).

Why C: it is the only option that gives agents something better than the human page with the chrome removed, and still keeps one source of truth per fact.

Open questions

1. Where are hand-written agent pages allowed? Only the home page and the loop, or any page? Too many overrides and the twin drifts from the human page.
2. Size budgets. llms.txt should stay small (a few KB; it is meant to be read whole). llms-full.txt for the whole compendium (docs(id): design plan, research, and n0 relay default #152) could run to megabytes (Fluree's is ~1.9 MB). Should we split it into llms-full.txt (levels 1 and 2) and a separate compendium file, or cap it and list the rest under ## Optional?
3. Twin URL shape: page.md or page.html.md? Both are allowed by the spec. page.md is shorter; page.html.md never collides with a source file.
4. Should the install prompt be versioned (pinned to the release it was published with), or always install latest?
5. Which harnesses does the prompt cover at first: only opencode, or all of Update Rust crate anyhow to 1.0.75 #48's list?

Ease · Impact · Related

Ease M (B is S once #117 is chosen; the install prompt and the in-binary docs make it M) · Impact high (agents are a primary audience of mcpx) · Depends on / related: #117, #118, #155, #158, #144, #145, #150, #152, #44 (closed, fixed in #53), #48, #49

Security/auth deferred; see tracker.