Skip to content

Cost Explorer ChatGPT-parity follow-ups: lane charts, chart extras, evidence cards, projection routes #1400

Open

Description

@dezren39

Follow-ups from the Cost Explorer ChatGPT-parity pass (branch feat/cost-explorer-chatgpt-parity). Everything in priorities 1–5 of the parity brief landed; the items below are what remains of the two ChatGPT explorers' feature sets, each with its blocker or reason noted. Nothing here blocks the parity PR.

Site 1 (aa-task-cost-explorer) remainders

  • Band-comparison lane charts — groups 3–6 (fastest per intelligence band, smartest per speed band, fastest per cost band, cheapest per speed target) render as lane charts on Site 1; the parity PR ships the tables. Add lane rendering over the same band computations.
  • Main-scatter comparison lines — smoothed upper-tail expectation, smoothed upper-envelope frontier, and the OLS cost–score trend line as toggleable overlays on the cost × score chart (the exact frontier and fitted-trend math is documented in Site 1's Data & methods tab).
  • Per-chart view settings — third-metric limit modes per chart (unset / at most / at least + thresholds, plus per-band limits), chart-label modes (key choices / all / none), and magnification (fit / 150% / 200%).
  • Unpriced-variant detail buttons under the main chart (configurations with a score but no published cost, listed as buttons opening the detail card). Partially covered today by chart 6 (scores-only bars) and the configurations table.
  • Per-band recommendation overrides ("Tune this band": per-band tradeoff overrides, absolute limits per band, band split/merge). The parity PR has global tuners only.
  • Provisional-estimate points + include-estimates toggle — blocked on measurement provenance in the Model data pipeline: workbook-free append-only event log + agent interpretation #1248 event log, same deferral as PR feat(pm): Cost Explorer — tabbed AA + Model explorer with frontier charts, perspectives, and full feature parity #1364's guest-estimate markers. Site 1's reviewed-data.json holds the estimate set.
  • Evidence & anecdote cards (Data & methods) — the notes/event log from Model data pipeline: workbook-free append-only event log + agent interpretation #1248 is the home for these; seed from Site 1's five cards when the notes surface exists.
  • AA directory table with OpenRouter ranks + OpenCode Console ids, and the OpenRouter/OpenCode coverage-check table (188 listings) — the fleet feed carries neither dataset today; needs a collector source before the page can render them.
  • Design references & historical snapshots section (Site 1's Data & methods) — historical artifacts; link the Drive/site originals rather than re-hosting.

Site 2 (drew-coding-model-explorer) remainders

  • Projection pricing routes — Site 2 prices the projection per route (Direct/benchmark API, OpenCode Console, OpenCode Go allowance). The parity PR prices from the recorded event-log rates only (one rate set per model). Needs per-route price events in the log before the selector means anything.
  • Cache-storage hours in the projection — no storage rate exists in the price event log; add the field to price events first.
  • Gemini promo vs Jan-2027 rates toggle — no promo windows recorded in price events yet.
  • Throughput / latency factor sliders for the projected time (Site 2's speed assumptions).
  • Shortlist side-by-side compare (up to 6) for catalog models — the parity PR has row detail cards and the recommendations shortlist; a pinned compare tray for the Model Explorer is the remaining piece.

Data notes

  • Site 1's linked datasets (reviewed-data.json, data.csv, CAI snapshots, coverage/cost/score audits, the CAI AInventory workbook) sit behind the ChatGPT login gate; retrieval needs the signed-in session or Aster's Drive upload (asked for in Slack #alignment). The parity PR proceeded from the signed-in browser inventory; pull the datasets when they land and diff the CAI observations against ainventory's benchmark_results as a cross-check.

Acceptance criteria

  • Each remainder item listed in the body is either implemented or explicitly deferred with a named follow-up issue: Site 1 band-comparison lane charts (groups 3-6), main-scatter comparison-line overlays, per-chart view settings, unpriced-variant detail buttons, and per-band recommendation overrides.
  • Lane charts render over the same band computations as the shipped tables (no divergent math).
  • Nothing in the parity PR (feat(pm): Cost Explorer — ChatGPT-site parity (CAI perspective, Pareto tables, recommendations, bands, projection) #1390) regresses: existing Cost Explorer pages/charts still render and their tests pass.

Activity

  1. developing-today commented on Oct 27, 2026

    @developing-today
    Contributor
  2. dezren39 commented on Oct 27, 2026

    @dezren39
    MemberAuthor

    One more Site 2 remainder for the record: standalone Terminal-Bench 4.0 / SciCode benchmark perspectives. Site 2 lets either benchmark drive its explorer directly. Here, Terminal-Bench survives as a CAI component score (observations table + detail cards) and SciCode is not carried by the fleet feed at all — a standalone-benchmark perspective needs feed data for both before it can be built.

  3. dezren39 commented on Oct 27, 2026

    @dezren39
    MemberAuthor

    Intake review

    Fleet monitor intake enrichment, 2026-10-27.

    Liveness: still live — this issue IS the remainder list from the merged parity PR #1390; no merged PR since implements these remainders (checked merged-PR titles this run).

    Notes: A bundle of independent follow-ups, not one change. Suggested approach: split per site/chart area into separate small PRs against the parity code from #1390, starting with the lane charts (same band math, rendering only — lowest risk). The comparison-line overlays need the frontier/trend math documented in Site 1's Data & methods tab.

    Risks: scope creep — the per-chart view-settings item is a settings system in miniature; sequence it last or split it out.

  4. added
    priority/normalOrdinary priority (the default)
    effort/lLarge: multi-day or open-ended; expect staged check-ins
    meta/epicEpic / bundle of related work
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Oct 27, 2026
  5. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    Fresh parity audit — 2026-10-31 (Drew: "the explorer is missing views and Controls the old ones had that were supposed to be ported")

    Re-audited the CURRENT live /aa page (aggregator on the laptop, feed status ok, 278 rankings / 608 catalog / 27 CAI rows) feature-by-feature against the 2026-10-27 signed-in inventory of the two ChatGPT explorers (~/workspace/ai-subscriptions/chatgpt-explorer-sites-inventory-2026-10-27.md). This issue remains the tracker; the table below is the current state, not the state at the parity PR.

    Structure note: the current page has TWO tabs (AA Explorer, Model Explorer) with the Site-1 views as in-page sections. Recommendations / Explore charts / Band comparisons / Pareto exist as sections; Data & methods does not exist at all (zero "methods" text on the page).

    Feature Old site Status Evidence
    Two perspectives (Coding agents / Intelligence & decode) S1 present toggle labels on live page
    CAI as first-class perspective + observations table S1 present (data partial) "Coding Agent Index observations" section renders; feed carries 27 CAI rows vs Site 1's 42 (ainventory holds 260 CAI v1.5 results — collector coverage gap, not a view gap)
    Pareto frontier tables w/ percentile ranks, both perspectives S1 present 4 tables on page (score-first / time-first / price-first / price–speed), percentile columns render
    Price-first percentile cutoff selector (50/60/75/none) S1 unverified tables render; selector not confirmed in this pass
    Recommendations: presets, global tradeoff tuners, cards, Pin, why-these-choices, shortlist CSV export S1 present sections + "Why these choices" + shortlist CSV download on page
    Per-band recommendation overrides ("Tune this band") S1 missing zero hits on live page; already a remainder above
    Band comparisons — six groups as tables S1 present "Band comparisons" section renders all groups
    Band-comparison LANE charts (groups 3–6) S1 missing tables only; already a remainder above
    Main-scatter comparison lines (smoothed upper tail, smoothed envelope, OLS trend) S1 missing envelope/smoothed/trend all zero on page; already a remainder above
    Chart settings: cost axis log/linear, near-leader window S1 present window controls render (near-leader ×5)
    Chart settings: frontier shape (stair/direct), size-by-measure, hide-dominated, recalculate-from-visible S1 missing all zero on live page
    Per-chart view settings (third-metric limit modes, label modes, magnification fit/150%/200%) S1 missing magnification zero; already a remainder above
    Inequality filters (metric × operator × value, AND/OR) S1 present filter builder on page; Pareto-gap metrics inside it not separately verified
    Advanced filters (labs/families multiselect, reasoning levels, search) S1 present filter labels render (Reasoning ×495, Labs, Families)
    Per-variant include list + chart click-to-toggle edit mode S1 unverified not confirmable from rendered HTML alone
    Saved setups (save/load/export/import/reset) S1 present Save setup control + setups.json export on page
    Provisional-estimate points + include-estimates toggle S1 missing zero hits; blocked on #1248 provenance, already a remainder above
    Unpriced-variant detail buttons S1 partial covered indirectly by scores-only chart + configurations table; already a remainder above
    AA directory table (OpenRouter ranks + OpenCode Console ids) S1 partial rankings/configurations tables render, but neither OR ranks nor Console ids are in the feed; already a remainder above
    OpenRouter/OpenCode coverage-check table (188 listings) S1 missing the page's "Coverage" section is fleet pool coverage, a different table; already a remainder above
    Data & methods: evidence & anecdote cards S1 missing no Data & methods view exists; already a remainder above
    Data & methods: methodology prose (relative-speed normalization, frontier math, caveats) S1 missing zero "method" text on the page
    Design references & historical snapshots section S1 missing already a remainder above
    Downloads: joined data JSON, visible-rows CSVs, shortlist CSV, setups JSON, chart PNG S1 present download functions on page
    Downloads: CAI snapshot JSONs, AA leaderboard snapshot, coverage/cost/score audit JSONs, XLSX, design ZIP S1 missing only the core set above is offered
    Task-assumptions projection (turns, in/answer/thinking tokens, cache share, presets, derived cost/time) S2 present "Task projection" section renders; Small patch / Feature / Repo task presets present
    DeepSeek peak vs off-peak toggle S2 present peak/off-peak controls render
    Projection pricing routes (Direct API / OpenCode Console / Go allowance) S2 missing zero hits; priced from event-log rates only; already a remainder above
    Cache-storage hours in projection S2 missing no storage rate in price events; already a remainder above
    Gemini promo vs Jan-2027 rates toggle S2 missing zero hits; already a remainder above
    Throughput / latency factor sliders S2 missing already a remainder above
    Score-per-dollar derived metric S2 present renders in projection/configurations
    Landscape scatter w/ switchable X axis (Model Explorer) S2 partial frontier scatters exist in the AA tab; no switchable-X landscape in the Model Explorer tab
    Shortlist side-by-side compare (up to 6, Model Explorer) S2 missing no compare tray; already a remainder above
    Site-2 benchmarks kept separate (Terminal-Bench / SciCode / AA) S2 unverified AA renders throughout; TB/SciCode presence not confirmed in this pass
    Expand + zoom/pan per chart — tracked separately code#3298 (filed + queued 2026-10-31) — not part of this issue

    Acceptance for the port (per missing item)

    Filed from the 2026-10-31 parity audit at Drew's direction. A queue task for this port is being enqueued against this issue.

  6. added
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    and removed on Nov 1, 2026
  7. 16 remaining items

  8. added
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  9. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: triaged (coverage pass)
    Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
    Found via: coverage pass 2026-11-03

  10. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
  11. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    Handed off — task 7689 (triage, lane code-f)

    Claimed by repo-worker-code (lane code-f); execution lane assigned opencode. Handed off to the opencode claimant to triage #1400 (Cost Explorer ChatGPT-parity follow-ups). Issue verified OPEN with no held/user-owned labels at handoff.

    • Task: task 7689
    • State: handed_off (execution_lane: opencode)
  12. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · findings

    Verdict: still live as an umbrella, and it should be split before more execution work. Fresh state: #1400 OPEN, effort/l, no held/user-owned labels. The first Site 1 checklist item is done: "Band-comparison lane charts" shipped in PR #3972 (feat(aggregator): add Cost Explorer band lane charts, merged 2026-11-02T15:45:33Z — the issue checklist still shows it unchecked).

    Ordering for the remainder (unblocked first):

    1. Unblocked, Site 1 UI-only (all read existing band/scatter computations): main-scatter comparison lines (smoothed upper-tail expectation, upper-envelope frontier, OLS trend — math documented in Site 1's Data & methods); per-chart view settings (third-metric limit modes, label modes, magnification); unpriced-variant detail buttons under the main chart; per-band recommendation overrides ("Tune this band"). Each is a bounded front-end slice and could ship independently.
    2. Link-only, no data dependency: design references & historical snapshots section (link Site 1 originals; do not re-host).
    3. Blocked — do not start: provisional-estimate points + include-estimates toggle and evidence/anecdote cards both wait on the Model data pipeline: workbook-free append-only event log + agent interpretation #1248 event log / notes surface; the AA directory table with OpenRouter ranks + OpenCode Console ids and the 188-listing coverage-check table wait on a collector source the fleet feed does not carry yet.
    4. Site 2 remainders (projection pricing routes and the rest of the Site 2 list) are independent of Site 1 and can be split on their own.

    Routing recommendation: carve one bounded small_fix/front-end packet per item in group 1 rather than one effort/l task for the umbrella; keep groups 3 and 4 attached to their blockers. No packet was enqueued from this triage run — the active task history around #1400 already contains earlier slice work (task 4908 → PR #3972), so a split decision should be made once, in a dedicated pass, instead of adding a duplicate live packet blind.

  13. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

effort/lLarge: multi-day or open-ended; expect staged check-inskind/enhancementmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/normalOrdinary priority (the default)source/creatorFiled by the new-issue-creator jobtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions