Skip to content

Agent tools need six registrations to work under subc, and nothing checks five of them #195

Closed

Description

@iceteaSA

A tool can be fully implemented, dispatch correctly, pass the whole suite, and still be dead under the daemon transport. I hit this twice in two hours after switching a box to subc, on a feature that had been green in CI for a month.

subc_plumbing_drift_test.rs already documents the class:

History: bash_wait_detach, bash_kill, bash_write, bash_notify and inspect_tier2_run each shipped in the plugins without a gate entry and were silently rejected (fail-closed) under the daemon transport until a log sweep caught the rejections in production.

Five tools, caught by a log sweep in production. That guard closes the hole for native plumbing sends. Agent tools have no equivalent, and there are more places to miss.

The six registrations

Standalone needs three: a main.rs dispatch arm, a translate_* arm, and the plugin tool. Subc needs three more, and one of the standalone three turns out to be gated by a fourth list:

# Location Miss symptom
1 is_subc_agent_core_tool (subc/manifest.rs:13) fail-closed rejection at subc/mod.rs:4031
2 command_lane (manifest.rs:~127) silent — _ => Lane::Mutating catches it, so a pure read serializes on the mutating lane
3 build_manifest (manifest.rs:~213) tool absent from the advertised manifest
4 BARE_TOOL_ORDER (subc-tool-schemas.ts) no schema emitted
5 subc_tool_schemas.json placeholder {"type":"object"} schema + a warn line
6 supports_tool (subc_translate.rs:831) silent, and it breaks standalone too

Six is the one worth attention. run_tool_call.rs:166 only calls the translate arm when supports_tool returns true:

let (command, translated_args) = if crate::subc_translate::supports_tool(bare_name) {
    ... subc_translate_owned_with_context(...)
} else {
    (bare_name.to_string(), map)   // passthrough, untranslated
};

A tool absent there has its translate_* arm as unreachable dead code, in every transport. Mine was, from the day I wrote it. It looked fine because the command handler duplicates the mode validation — but translation is the only layer that calls resolve_path_from_project_root, and the error shapes diverged. Same input, before and after adding one name to that list:

question:""  before → {"code":"search_failed","message":"aft_gather_context: search failed: {\"code\":\"invalid_request\",\"message\":\"query must be non-empty\"}"}
question:""  after  → {"code":"invalid_request","message":"aft_gather_context: provide either 'question' or 'symbol'+'filePath'"}

The inner search error was leaking out under a wrong code, because the handler validates with is_some() presence checks and an empty string passed its mode gate. It also accepted question plus a lone symbol and silently dropped the symbol.

Why the existing tests can't see any of this

Every guard iterates a hand-maintained list instead of deriving one:

  • build_manifest_serves_embedded_tool_schemas and build_manifest_classifies_execution_mode_by_observable_effect both iterate build_manifest()'s own output — a tool missing from the manifest is structurally invisible to them
  • supports_tool_covers_every_translated_arm iterates a hardcoded 21-name array. Its comment says it exists so that "if a translate arm is added but the allowlist isn't updated, that tool would silently bypass translation" — exactly my bug, and it passed, because the array omitted the tool
  • PLUGIN_NATIVE_SENDS is a const &[&str] with "Keep in sync when adding plugin bridge calls" — the guard against hand-maintained-list drift is a hand-maintained list, and it covers only native plumbing, not agent tools
  • the v0.49 surface audit reads subc_tool_schemas.json for legacy vocabulary only
  • registration-parity.test.ts does compare plugin surfaces, which is why my tool needed a HOSTONLY-V049-* inventory row — but it never looks at the subc manifest

The suggestion

subc_tool_schemas.json is already the cross-language bridge: generated from the real plugin tool factories by build-tool-schemas.ts, byte-pinned by a freshness test, and include_str!-embedded in the Rust manifest. It can be the source of truth instead of a sixth thing to remember.

One Rust test, reading the embedded map, asserting for every key:

  • is_subc_agent_core_tool(name)
  • present in build_manifest().provides[ToolProvider].tools
  • command_lane(name) was matched explicitly rather than falling through to _ => Lane::Mutating
  • supports_tool(name)

That turns six registrations into two — add to BARE_TOOL_ORDER, regenerate — and the plugin-side existence assertion in buildSubcToolSchemas() already fails loudly if a factory rename breaks the wiring. It would have caught both of my bugs at commit time, and all five tools in the drift guard's history.

The lane check needs command_lane to distinguish "matched Mutating" from "fell through to Mutating". A command_lane_explicit(&str) -> Option<Lane> with the current function as a wrapper is the smallest way there, though you may prefer a different shape.

Happy to implement it if you want it — say which shape you'd accept. Also happy to leave it; you know the invariants here better than I do, and this is a report either way.

Reproduction context: found on a box running aft as a subc module across two sessions, a config repo and this one. The consumer-side session can add detail if useful.

Activity

  1. iceteaSA commented on Aug 8, 2026

    @iceteaSA
    ContributorAuthor

    Consumer-side reproduction, as offered (main harness seat, same box, real opencode session over the subc transport — not CLI probes):

    Bug 1 (manifest gate): first aft_gather_context call after the daemon flip returned tool "gather" is not in the AFT tool manifest — fail-closed at the gate, correct behavior, wrong manifest. Every other hoisted tool (search/zoom/outline/bash/read/edit) worked in the same session, which is what isolated it to registration rather than transport.

    Bug 2 (unreachable translation): after the manifest fix, the A/B from a live session: question:"" on the old binary leaked the inner error through a wrong layer (search_failed: query must be non-empty); on the fixed binary it returns the translation-layer shape (invalid_request: provide either 'question' or 'symbol'+'path'). The new error text IS the proof translate_gather is now on the live tool_call path.

    Timeline datum for the derived-set argument: both bugs were invisible to a month of green CI and 13 rebases in standalone mode, and both surfaced within ~2 hours of the first production session on the daemon transport — the first on the first gather call, the second on the first adversarial input probe.

  2. iceteaSA commented on Aug 8, 2026

    @iceteaSA
    ContributorAuthor

    Seventh and eighth instances, found the hard way an hour after filing this — my own PR went red on CI for exactly the failure mode described above.

    crates/aft/tests/integration/subc_bridge_test.rs:6886
      assert_eq!(tools.len(), 21, "expected 21 manifest tools");
    
    crates/aft/src/subc/manifest.rs:279
      const CORE_TOOLS: [&str; 21] = [ … ];
    

    Three jobs red (Unit Linux, Unit macOS, Cargo test Windows). Both are hardcoded counts that had to change when a 22nd tool appeared, and neither is reachable from the registration sites — so adding a tool correctly at all six points still breaks the build.

    CORE_TOOLS is worth a closer look, because it is the blind list from the original report. build_manifest_serves_embedded_tool_schemas and build_manifest_classifies_execution_mode_by_observable_effect both iterate it. A tool absent from CORE_TOOLS is not checked by either test — but the array's declared length [&str; 21] is a compile-time assertion, so it fails the build rather than passing silently. That is accidental protection: the count catches you, the coverage does not. Add a tool to build_manifest and to CORE_TOOLS and both tests genuinely cover it; add it to build_manifest alone and you get a compile error that tells you nothing about why.

    The count now reads 22 in both places on my branch. That is a correct-by-construction update, not a workaround — the tool count genuinely changed — but it means eight sites now have to move in lockstep for one tool, and the two that fail loudest are the two that check the least.

    Deriving from subc_tool_schemas.json keys, as proposed above, subsumes both: tools.len() becomes the key count, and CORE_TOOLS becomes the key set. No hand-maintained number survives.

    One more datum for the "silent for a month" argument, this one about the local gate rather than the code. cargo-nextest cancels the run on first failure. My box has two permanent environmental failures (sandbox_native_test, from a /home/linuxbrew path on PATH) at positions ~1317 of 1642, so every default gate run stops there and never reaches subc_bridge_test at all. My local run reported Summary [26.469s] 1330/1642 tests run: 1328 passed, 2 failed and I read it as baseline. The failing test executes in 0.11s once actually reached. --no-fail-fast is now my default, but a contributor whose environment produces any early failure has a gate that silently stops covering the second half of the suite — worth knowing independently of this issue.

  3. ualtinok commented on Aug 9, 2026

    @ualtinok
    Collaborator

    This is the right diagnosis and the right fix. The six-site registration is real, the guards you name are exactly as weak as you describe — the supports_tool array guard can only defend names it already knows, which is backwards — and your timeline (a month green in standalone, dead in two hours under the daemon transport) is the strongest argument for deriving from the artifact instead of maintaining a sixth list.

    One verification we ran before accepting, since it's the load-bearing claim: the generator does not have its own emit list. buildSubcToolSchemas() calls the real tool factories and destructures each tool by name with a throw on absence, so a factory rename fails the build loudly. BARE_TOOL_ORDER remains the single hand-maintained selection list, which is exactly your "two registrations" residue: one visible list, loudly coupled to the runtime it selects from. Strictly better than six independent ones.

    Accepting your proposal with the review bar pinned:

    1. One Rust test beside build_manifest, iterating the include_str!-embedded subc_tool_schemas.json keys, asserting per key: is_subc_agent_core_tool, manifest presence, command_lane_explicit(name).is_some(), and supports_tool.
    2. command_lane_explicit(&str) -> Option<Lane> with the current command_lane as a thin wrapper — your suggested shape is fine.
    3. Deliberate exceptions (a tool that intentionally skips translation) go in an explicitly documented exemption const in the test — empty today — so a future exemption is a reviewed decision, not a silent skip.
    4. CORE_TOOLS and the hardcoded tool counts derive from the same keys: count = key count, and the test asserts CORE_TOOLS equals the key set.
    5. The derived test asserts its own denominator — key count equals BARE_TOOL_ORDER's length — so a generation defect can't read as a green zero-case suite.

    Your fail-fast observation is taken separately and was confirmed on our side the same day you filed this (a gate stopped at the first failure and hid three more behind it); our gate scripts moved to --no-fail-fast.

    PR welcome — the five points above are the review bar. And for the report quality: the before/after error shapes and the consumer-side reproduction made this a one-read accept.

  4. ualtinok commented on Aug 9, 2026

    @ualtinok
    Collaborator

    Closed by #200 — the derived test is on main and is now the tripwire the hashline campaign's registration acceptance names explicitly. Both judgement calls in the PR (slice-typed CORE_TOOLS, Rust-side denominator against the manifest count with the TS half pinned by the existing byte-for-byte artifact test) are right by the review bar's own logic: no arity literals, both halves of the cross-language denominator asserted where each language can see it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions