Skip to content

[FEATURE]: Structured usage export for task-level evaluations #48821

Open

Description

@devjaime

Feature hasn't been suggested before

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

I want to compare the same coding task with and without an external retrieval tool. Session totals do not identify the experiment, distinguish missing usage from zero, or establish equivalent task quality.

Would a small structured usage export, reusing the existing session transcript accounting, fit the project? The first version would summarize each provider step into disjoint input, output, reasoning, cache-read, and cache-write buckets, report its accounting scope, and support JSON or CSV. Graph retrieval stays in an external MCP integration.

Related work: #42324 covers multi-step message totals, #45417 covers subagent rollups, #46371 covers hidden calls, and #41915 proposes a human-facing usage report. #47822 proposes returning child-session metrics from the task tool; this proposal instead describes the accounting scope of a complete exported session. #9387 asks for transcript formats; this proposal exports a machine-readable usage summary rather than another transcript representation.

Before making the PR ready, I would like to confirm whether the export belongs under opencode export and how incomplete child/fork usage should be represented.

Activity

  1. github-actions commented on Sep 13, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    (You have already referenced #42324, #45417, #46371, and #41915 — good.)

  2. opencodejr commented on Oct 28, 2026

    @opencodejr

    Thanks for the detailed proposal, @devjaime! As I understand it, you'd like a machine-readable (JSON/CSV) usage export for a session. Each provider step would be split into disjoint input, output, reasoning, cache-read and cache-write buckets, and the export would say what accounting scope it covers, so the same task can be compared with and without an external retrieval tool.

    The maintainers will consider it, including your questions about whether it belongs under opencode export and how incomplete child/fork usage should be represented. I can't promise anything about whether or when it will be picked up.

    Related issues I found:

    Until then, you can get most of this outside core by reading the session's messages through the SDK, or by post-processing the JSON from opencode export.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions