Skip to content

Include cached token metrics in Integration Test Average Token Usage dashboard panel - #2189

Merged
JasonYeMSFT (JasonYeMSFT) merged 2 commits into
mainfrom
copilot/update-integration-test-average-token-usage
May 8, 2026
Merged

Include cached token metrics in Integration Test Average Token Usage dashboard panel#2189
JasonYeMSFT (JasonYeMSFT) merged 2 commits into
mainfrom
copilot/update-integration-test-average-token-usage

Conversation

Copilot AI commented May 7, 2026

Copy link
Copy Markdown
Contributor

The Integration Test Average Token Usage panel only surfaced input/output/total tokens, which hid cache-driven cost signals. This update adds cached token counts to the panel’s per-test averages and display so token usage is more representative.

  • Dashboard token aggregation

    • Extended integration token usage row shaping to include per-run averages for:
      • cacheReadTokens
      • cacheWriteTokens
    • Kept existing sorting behavior by average totalTokens.
  • Panel table rendering

    • Updated the token column header from In / Out / Total to:
      • In / Out / Cache Read / Cache Write / Total
    • Rendered cache read/write values inline with existing token metrics for each test row.
  • Styling + regression coverage

    • Added token text styling for new cache token spans in the dashboard CSS.
    • Added a focused dashboard test to assert:
      • cached token averages are included in row construction
      • cache token fields/header are rendered.
rows.push({
  inputTokens: Math.round((usage.inputTokens || 0) / runCount),
  outputTokens: Math.round((usage.outputTokens || 0) / runCount),
  cacheReadTokens: Math.round((usage.cacheReadTokens || 0) / runCount),
  cacheWriteTokens: Math.round((usage.cacheWriteTokens || 0) / runCount),
  totalTokens: Math.round((usage.totalTokens || 0) / runCount),
});

Copilot AI changed the title [WIP] Update integration test average token usage to include cached counts Include cached token metrics in Integration Test Average Token Usage dashboard panel May 7, 2026
@tmeschter
Tom Meschter (tmeschter) marked this pull request as ready for review May 7, 2026 21:08
Copilot AI review requested due to automatic review settings May 7, 2026 21:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the dashboard’s “Integration Test Average Token Usage” panel so per-test averages include cache read/write token counts, improving visibility into cache-driven costs during integration runs.

Changes:

  • Extended integration token usage row construction to compute cacheReadTokens and cacheWriteTokens per-run averages.
  • Updated the panel rendering to display In / Out / Cache Read / Cache Write / Total and added new DOM spans for the cache metrics.
  • Added CSS styling for the new cache token spans and introduced a focused dashboard test asserting the new fields/header are present.
Show a summary per file
File Description
scripts/src/dashboard/tests/integration-token-usage.test.ts Adds a regression test that asserts cached token metrics are included in row construction and rendered in the table.
dashboard/assets/style.css Extends muted styling to the new cache token metric spans.
dashboard/assets/dashboard.js Computes per-test cached token averages and renders cache read/write alongside existing token metrics in the panel.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 2

Comment thread dashboard/assets/dashboard.js
Comment thread scripts/src/dashboard/__tests__/integration-token-usage.test.ts
@JasonYeMSFT
JasonYeMSFT (JasonYeMSFT) merged commit fee8d9c into main May 8, 2026
13 checks passed
@JasonYeMSFT
JasonYeMSFT (JasonYeMSFT) deleted the copilot/update-integration-test-average-token-usage branch May 8, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Integration Test Average Token Usage

5 participants