Skip to content

Structured errors/warnings (with tips) in JSON output for agents #1159

Open

Description

@theoephraim

Agents reading varlock load output as JSON currently can't see warnings or the actionable tips attached to errors and warnings.

  • --format json-full only carries errors, as errors.root (strings) and errors.configItems (item key -> ;-joined message string). No warnings, tips, or locations.
  • --agent (redacted key/value map) has no place for problems at all.
  • The pretty stderr output has everything, but it's meant for humans.

The tips are often the most useful part for an agent (e.g. "mark it @sensitive=false, or add @type=string"), and a location tells it exactly which line to edit.

Constraints

  • Plain json-full is consumed programmatically (framework integrations, and piped into __VARLOCK_ENV blobs in the sandbox guides), so it should stay lean and stable. We briefly added a warnings object to it in Fix duplicate and missing error output #1156 and reverted that for this reason.
  • getSerializedGraph also produces the injected __VARLOCK_ENV blob via run, so any additions there should be opt-in.

One possible shape

Gate a richer report on --agent, e.g. --agent --format json-full adds a flat diagnostics array:

{ "severity": "warning", "item": "FOO", "source": ".env.schema",
  "message": "sensitive, but a number is never redacted",
  "tip": "If it is not a secret, mark it `@sensitive=false`...",
  "location": { "file": ".env.schema", "line": 3, "column": 1 } }

A flat list is easier for an agent to filter than nested joined strings. It should follow the same item visibility rules as config (no @internal / filtered-out / _VARLOCK_* items), and messages/tips must never contain sensitive values.

Open questions

  • Behind --agent on json-full, a dedicated format, or something else (e.g. a varlock doctor/check --agent style command)?
  • Should errors in plain json-full stay as-is for back-compat even if agents get the richer form?
  • Include plugin warnings (currently only printed via showPluginWarnings)?
  • Update guides/ai-tools.mdx and the varlock skill to point agents at it.

Activity

  1. pullfrog commented on Sep 29, 2026

    @pullfrog
    Contributor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions