Skip to content

Workflows 2/5: reference workflows — review loop (evaluator–optimizer) and routing #82

Closed

Description

@prometheusgr

Part of #80. Slice 2 of 5 (v1). Depends on slice 1.

Context

Slice 1 ships the primitives and re-expresses the single-agent chat as a workflow. This slice ships the reference multi-agent workflows forkers will copy. They are both examples and the tests that prove the primitives compose.

Scope

  1. Interpret → work → review loop (evaluator–optimizer): an interpreter restates the request as a task spec; a worker completes it (with search_knowledge_base); a reviewer returns structured output ({approved: bool, issues: [...]}) and the worker revises until approved or max_iterations is reached. Structured review output is required, not prose: local models' free-text critiques are unreliable and code can't branch on them. Use llama.cpp JSON-schema / grammar-constrained output.
  2. Route to specialists (routing): a classifier picks one of two specialist agents.
  3. State shape decision, documented: shared task/draft/review fields in workflow state, with only the final answer in messages, so the reviewer's comments don't pollute the worker's context and aren't shown to the user by default.
  4. Structural tests for each reference workflow via the scripted fake harness (reviewer rejects once, loop hits the cap, router picks each branch).
  5. Manual verification against the real stack (docker-compose up --profile llm), with a note on observed latency per turn (each hop is a full CPU inference).

User-visible signal

For forkers: two copyable reference workflows. For end users: none until slice 5 (workflow selection). Call this out in the PR.

Configuration call-out

Per-workflow max_iterations lives in YAML (bounded by slice 1's ceiling). No new SystemSettings fields.

Definition of Done

  • Both reference workflows as built-in YAML, with structural tests.
  • Reviewer output schema enforced; malformed reviewer output is handled (treated as not approved, counted against the cap) and tested.
  • Real-model manual run documented in the PR, including latency.
  • Docs: worked examples in the workflow guide.
  • Standard merge gate from AGENTS.md's Definition of Done applies in full.

Activity

  1. added
    type:featureBrand-new capability that doesn't exist today
    effort:LLarge: full-stack slice (schema+backend+frontend+e2e), roughly a phase-sized chunk
    area:agent-toolsLangGraph agents and MCP tool integrations
    on Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:agent-toolsLangGraph agents and MCP tool integrationseffort:LLarge: full-stack slice (schema+backend+frontend+e2e), roughly a phase-sized chunkpriority:P2Valuable, not urgenttype:featureBrand-new capability that doesn't exist today

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions