Skip to content

Expose Auto tier switching across all six SDKs - #2514

Merged
SteveSandersonMS merged 7 commits into
mainfrom
issue-18045-expose-auto-tier-switching-across-sdks
Sep 4, 2026
Merged

Expose Auto tier switching across all six SDKs#2514
SteveSandersonMS merged 7 commits into
mainfrom
issue-18045-expose-auto-tier-switching-across-sdks

Conversation

@andyfeller

@andyfeller andyfeller commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Why

Copilot CLI 1.0.83-4 added Auto tier switching, which lets a session steer auto model routing toward efficiency, balance, or intelligence. The runtime exposes it as session.model.switchAutoTier, but none of the SDKs surface it, so integrators building model pickers cannot offer the control at all.

This builds on #2437, which added the Auto tier option at session creation. That covered choosing a tier up front; this covers changing it afterward.

What the runtime actually does

This matters because it shapes the API. The runtime does not apply a preference when you ask for it. It records the request and commits it only when a later user turn using the auto model successfully obtains a model and token pair from the provider. A pending response confirms the request was accepted, not that it took effect.

Two consequences the SDK surface has to respect:

  • switchAutoTier is rejected unless auto is the selected model. The error is Auto preference switching requires the selected model to be 'auto'.
  • Setting a model and setting a tier are separate operations, because only the tier is deferred.

What changed

Each SDK gains two things:

  1. setAutoTier(tier) — stage a preference without touching the selected model.
  2. An Auto tier option on setModel — stage a preference atomically with selecting auto.
SDK Stage a preference Option on setModel
Node.js setAutoTier(tier | null) autoTier?: AutoTier | null
Python set_auto_tier(tier | None) auto_tier= (unset / value / None)
Go SetAutoTier(ctx, *AutoTier) AutoTier *AutoTier + ResetAutoTier bool
.NET SetAutoTierAsync(AutoTier?) AutoTier? + ResetAutoTier
Rust set_auto_tier(Option<AutoTier>) AutoTierPreference::{Tier, Reset}
Java setAutoTier(AutoTier) setAutoTier(...) / setResetAutoTier(true)

The design decision worth reviewing

setModel needs three outcomes, not two:

Call Effect on a staged preference
tier omitted preserved
explicit tier replaced
explicit reset cleared

This preserves omission semantics — an existing caller passing no tier keeps behaving exactly as before, which is why the third state cannot simply reuse "null."

Languages that model absence and null separately express it directly: Node uses undefined versus null, Python uses a sentinel default versus None, and Rust uses an AutoTierPreference sum type. Go, .NET, and Java have no way to distinguish "not set" from "set to null" on an options object, so they carry a separate reset flag and reject requests that set both.

I recognize this adds a public concept that does not exist in the wire protocol, and that inventing surface to work around a language limitation deserves scrutiny. The alternative — dropping the reset capability in three languages, or making setModel silently clear a staged preference — seemed worse. I'm open to a different shape.

The whole surface is marked experimental in every language, so we can change it.

How this was checked

Unit tests assert the JSON-RPC payload each SDK builds, including that a reset sends an explicit null rather than omitting the field.

More importantly, 12 new end-to-end tests run against a real 1.0.83-4 runtime and read state back through model.getCurrent(), so every assertion reflects recorded runtime behavior rather than the payload we serialized. Two scenarios, identical in all six SDKs:

Staging and resetting:

Call Response pendingAutoTier after
setAutoTier("efficiency") pending efficiency
setAutoTier("intelligence") pending, superseded efficiency intelligence
setAutoTier(null) unchanged, superseded intelligence none

Omission semantics:

Call pendingAutoTier after
setAutoTier("balance") balance
setModel("auto"), tier omitted balance
setModel("auto", tier: "intelligence") intelligence
setModel("auto", reset) none

Both snapshots list auto and record no conversation, since these are pure RPC exchanges. Covering the commit path through session.model_change and session.auto_tier_switch_failed requires a recorded conversation and is left as follow-up.

Suites run locally on all six languages: Node.js typecheck and tests, Python ruff and pytest, Go build/vet/test, .NET (798 passed), Rust, and Java (mvn verify, 25 integration tests including AutoTierIT).

One pre-existing failure is unrelated to this change: TestMultiClientE2E/one_client_approves_permission_and_both_see_the_result fails identically on a pristine origin/main checkout.

Reading the diff

42 files, no generated code. The bulk is tests and per-language documentation:

Portion Files
Tests 18
Documentation (six READMEs plus one guide) 7
Hand-written SDK code 15
Snapshots 2

The two new snapshots are four lines each.

andyfeller and others added 5 commits September 3, 2026 21:46
The runtime now accepts an Auto routing preference change on a live
session through `session.model.switchAutoTier`, and reports the
authoritative committed, pending, and activating preferences through
`session.model.getCurrent`. Without SDK support, integrators could only
choose a tier at session creation or resume and had no way to change it
or observe whether a change took effect.

Each SDK gains two capabilities:

* `setAutoTier` changes the routing preference without changing the
  selected model.
* The model switch options gain an Auto tier field, which stages a tier
  atomically with selecting `auto`.

The runtime distinguishes an explicit null tier, meaning "return to
provider-default routing," from an absent one, meaning "leave the
preference alone." Python, Go, .NET, Java, and Rust generated wrappers
drop null properties, so those SDKs build the request payload directly
where a null must survive; each bypass carries a comment explaining why.
Node.js needed no workaround. The tri-state choice is expressed
idiomatically per language: `undefined`/tier/`null` in Node.js, an
`_UNSET` sentinel in Python, a `ClearAutoTier` flag validated as
mutually exclusive in Go, .NET, and Java, and an `AutoTierPreference`
enum in Rust.

Tests cover the wire payloads for both methods, including the explicit
null case, and decoding of the ephemeral
`session.auto_tier_switch_failed` event across all four failure reasons
plus the null requested-tier case.

Documentation previously stated that a resident resume rejects a
different tier. The authoritative schema now says the runtime requests a
safe switch applied after the resume succeeds, which cannot change a
turn already in flight. This corrects that text in all six SDKs and in
the session persistence guide.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
Reviewing the change against the standards maintainers have applied to
earlier SDK pull requests surfaced four issues worth fixing before
opening the pull request.

Python could not accept the enum it hands back. `set_auto_tier` and
`set_model` were typed for the `AutoTier` string literal, but results and
events carry the generated `AutoTier` enum. Feeding that value back
raised `TypeError: Object of type AutoTier is not JSON serializable`,
because the JSON-RPC encoder calls `json.dumps` with no enum support.
Both methods now accept either representation and normalize to the wire
value. Added a regression test.

.NET and Go hand-copied request fields. Both built a second, partial copy
of the model-switch payload so an explicit null tier would survive
serialization. Each copy listed only a subset of the generated request's
fields, so a new field on the schema would have been silently dropped on
the clear path. Both now serialize the generated request and write the
null back, matching what Rust, Java, and Python already did. This also
removes two hand-written request types and their serializer
registrations from the .NET SDK.

Java allocated raw `ObjectMapper` instances instead of using the shared
`MAPPER`, which is configured to match the JSON-RPC encoder. A
differently configured mapper could produce a different wire shape.

The .NET test compared the status enum through `ToString`; it now
compares the enum directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
The .NET and Java implementations carried [Experimental] and
@CopilotExperimental, but the Node.js, Python, Go, and Rust equivalents
carried no marker. That asymmetry is the exact gap maintainers have
flagged before: C# gets the annotation and the other languages are left
without the corresponding doc annotation.

Auto tier routing is still moving — the `fast` tier is not yet exposed
and the runtime contract may shift — so the surface is marked
experimental consistently in all six SDKs, each using the convention
already established in that language.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
The SDKs described the same idea three different ways: Rust called it
`AutoTierPreference::ProviderDefault`, while Go, .NET, and Java called it
`ClearAutoTier`. Cross-SDK consistency is the standard maintainers apply
most often, and a reader comparing two SDKs had no way to tell these were
the same operation.

Everything now uses "reset": `ResetAutoTier` in Go and .NET,
`setResetAutoTier` in Java, and `AutoTierPreference::Reset` with
`with_reset_auto_tier` in Rust.

The underlying shapes stay language-idiomatic. Node.js and Python express
all three states natively, because `null` and `None` are distinguishable
from an omitted argument. Rust uses a sum type. Go, .NET, and Java cannot
make that distinction in a single value, so they keep a separate reset
option. Forcing Rust's enum onto Go would mean exported constructor
functions in place of the pointer-and-flag pattern Go already uses for
"unset versus explicit zero", which trades a cross-language inconsistency
for a within-language one.

Documented the trade-off and added a per-SDK table for staging a tier on
a model switch, so the difference is explained rather than discovered.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
The unit tests for Auto tier switching only check the JSON-RPC payload the SDK
builds. That proves the wire shape but not that the runtime interprets it the
way the API documents, which is the part reviewers have asked us to demonstrate
on earlier model-switching work.

These tests run against a real Copilot runtime and read the staged state back
through `model.getCurrent()`, so every assertion reflects recorded runtime
behavior. Two scenarios, identical across all six SDKs:

- Staging and resetting. A request is accepted as `pending`, a second request
  replaces the first and reports the tier it displaced, and a null tier returns
  the session to provider-default routing.
- Omission semantics. Calling `setModel("auto")` without a tier preserves the
  staged preference, passing a tier replaces it, and asking for a reset clears
  it. These are three distinct outcomes, which is why the reset is expressed
  separately from the tier value in every language.

The runtime only commits a preference on a later turn that uses the `auto`
model, so it rejects `switchAutoTier` unless `auto` is selected. Both snapshots
therefore list `auto` and record no conversation; covering the commit path
through `session.model_change` and `session.auto_tier_switch_failed` needs a
recorded conversation and is left as follow-up.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
Copilot AI balanced review requested due to automatic review settings September 4, 2026 02:30
@andyfeller
andyfeller requested a review from a team as a code owner September 4, 2026 02:30
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The Rust source-breaking API change and missing experimental gating in .NET and Java must be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 High severity · 2 Medium severity · 1 Low severity

New issues introduced by this change (4)
Severity Finding
High severity rust/​src/​types.rs — Adding this field to the existing exhaustively constructible public struct is a source-breaking API…
Medium severity dotnet/​src/​Types.cs — These new options are not marked with the SDK's [Experimental(Diagnostics.Experimental)]
Medium severity java/​sdk/​src/​main/​java/​com/​github/​copilot/​rpc/​SetModelOptions.java — This new options type exposes the setModel Auto-tier path without @&ZeroWidthSpace;CopilotExperimental, while…
Low severity rust/​tests/​e2e/​auto_tier.rs — Rust SDK comments should not reference another SDK as their canonical counterpart. Keep this test's…
What changed in this PR

Exposes live Auto-tier switching across all six SDKs while preserving omitted, selected, and reset preference semantics.

Changes:

  • Adds setAutoTier APIs and setModel Auto-tier options.
  • Exposes supporting state, status, and failure-event types.
  • Adds unit, E2E, snapshot, and documentation coverage.
File Summary / Review
test/​snapshots/​auto_tier/​should_stage_and_reset_auto_tier_preference.yaml Adds staging/reset fixture.
test/​snapshots/​auto_tier/​should_preserve_auto_tier_when_set_model_omits_it.yaml Adds omission-semantics fixture.
rust/​tests/​e2e/​auto_tier.rs Tests Rust runtime behavior. Nit (2 votes): Make the test description self-contained rather than referencing the Node.js SDK.
rust/​tests/​e2e.rs Registers Rust E2E tests.
rust/​tests/​api_types_test.rs Tests Rust wire types.
rust/​src/​types.rs Adds Rust tier preferences. Critical (2 votes): Adding auto_tier to the existing constructible public struct breaks downstream struct literals; use an additive compatible API or handle this as a breaking release.
rust/​src/​session.rs Implements Rust switching APIs.
rust/​README.md Documents Rust usage.
python/​test_event_forward_compatibility.py Tests Python failure events.
python/​test_client.py Tests Python RPC payloads.
python/​README.md Documents Python usage.
python/​e2e/​test_auto_tier_e2e.py Tests Python runtime behavior.
python/​copilot/​session.py Implements Python switching APIs. Nit (1 vote): Mark the set_model Auto-tier path as experimental in its public documentation.
python/​copilot/​client.py Shares Python Auto-tier typing.
python/​copilot/​__init__.py Exports Python result types.
nodejs/​test/​session-event-types.test.ts Tests Node.js event exports.
nodejs/​test/​e2e/​auto_tier.e2e.test.ts Tests Node.js runtime behavior.
nodejs/​test/​client.test.ts Tests Node.js RPC payloads.
nodejs/​src/​types.ts Exports Node.js RPC types.
nodejs/​src/​session.ts Implements Node.js switching APIs.
nodejs/​src/​index.ts Exposes Node.js public types.
nodejs/​README.md Documents Node.js usage.
java/​sdk/​src/​test/​java/​com/​github/​copilot/​SessionAutoTierSwitchTest.java Tests Java wire payloads.
java/​sdk/​src/​test/​java/​com/​github/​copilot/​SessionAutoTierEventTest.java Tests Java failure events.
java/​sdk/​src/​test/​java/​com/​github/​copilot/​AutoTierIT.java Tests Java runtime behavior.
java/​sdk/​src/​main/​java/​com/​github/​copilot/​rpc/​SetModelOptions.java Adds Java model-switch options. Moderate (2 votes): Apply @CopilotExperimental consistently to this API and CopilotSession.setModel(SetModelOptions). Nit (1 vote): Clarify that model is required.
java/​sdk/​src/​main/​java/​com/​github/​copilot/​rpc/​CapiSessionOptions.java Updates Java lifecycle documentation.
java/​sdk/​src/​main/​java/​com/​github/​copilot/​CopilotSession.java Implements Java switching APIs.
java/​README.md Documents Java usage.
go/​types.go Updates Go lifecycle documentation.
go/​session.go Implements Go switching APIs.
go/​session_test.go Tests Go RPC payloads.
go/​session_event_serialization_test.go Tests Go failure events.
go/​README.md Documents Go usage.
go/​internal/​e2e/​auto_tier_e2e_test.go Tests Go runtime behavior.
dotnet/​test/​Unit/​SessionEventSerializationTests.cs Tests .NET failure events.
dotnet/​test/​Unit/​ClientSessionLifetimeTests.cs Tests .NET RPC payloads.
dotnet/​test/​E2E/​AutoTierE2ETests.cs Tests .NET runtime behavior.
dotnet/​src/​Types.cs Adds .NET model-switch options. Moderate (2 votes): Add the SDK’s experimental attribute to both new properties.
dotnet/​src/​Session.cs Implements .NET switching APIs.
dotnet/​README.md Documents .NET usage.
docs/​features/​session-persistence.md Documents cross-SDK lifecycle behavior. Nit (1 vote): Correct the reset-state explanation because Rust uses AutoTierPreference::Reset; only Go, .NET, and Java use reset flags. Nit (1 vote): State that live switching requires Copilot CLI 1.0.83-4 or later.
Suppressed comments (4)

docs/features/session-persistence.md:312

  • This count and explanation are incorrect: Rust represents the third state with AutoTierPreference::Reset, while only Go, .NET, and Java use a separate reset flag. As written, the guide contradicts both the table above and the PR's API design summary.
Node.js and Python express all three states natively, because `null`/`None` is distinguishable from an omitted argument. The other four SDKs cannot make that distinction in a single value, so they carry a separate reset option. Omitting both always means "leave the current preference alone."

docs/features/session-persistence.md:274

  • Please state that live tier switching requires Copilot CLI 1.0.83-4 or later. The only version requirement currently visible in this section is 1.0.82-1 for creation/resume, so users can reasonably infer that the new RPC works on that older runtime and instead receive an unknown-method failure.
Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing.

java/sdk/src/main/java/com/github/copilot/rpc/SetModelOptions.java:11

  • This says every option is optional, but model is required and CopilotSession.setModel throws when it is absent. Clarify the requirement so callers do not rely on the documented omission behavior.
 * All setter methods return {@code this} for method chaining. Every option is
 * optional; an unset option leaves the corresponding session state unchanged.

python/copilot/session.py:3109

  • The auto_tier path on set_model is part of the new experimental surface, but unlike set_auto_tier its public documentation does not mark it experimental. This makes the stability contract inconsistent with the PR description and the adjacent API; add the same experimental warning here.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread rust/src/types.rs
Comment thread dotnet/src/Types.cs
Comment thread java/sdk/src/main/java/com/github/copilot/rpc/SetModelOptions.java
Comment thread rust/tests/e2e/auto_tier.rs Outdated
andyfeller and others added 2 commits September 3, 2026 23:25
The Auto tier README example is a fragment that references an undefined
'session' binding, so rustdoc could not compile it as a doctest. Mark it
'rust,ignore' to match every other fragment in the file, and apply the
rustfmt import merge the format job asked for.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
Gate the experimental Auto tier options consistently. The .NET
SetModelOptions.AutoTier and .ResetAutoTier properties, the Java Auto tier
accessors, and the Python set_model auto_tier parameter now carry the same
experimental marker their dedicated setAutoTier counterparts already had.

Correct two documentation errors. The reset-state paragraph claimed four SDKs
cannot express reset in a single value; Rust can, through
AutoTierPreference::Reset, so only Go, .NET, and Java need a separate flag.
The live-switching section now states that it needs Copilot CLI 1.0.83-4,
which is newer than the 1.0.82-1 required to select a tier at create or resume.

Fix the Java class Javadoc, which said every option is optional when setModel
rejects a null model, and rewrite the Rust E2E doc comment to describe what it
covers instead of pointing at the Node.js test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2aad36e9-e078-4b9c-afa1-0c7cd2c71f74
@github-actions

This comment has been minimized.

@SteveSandersonMS SteveSandersonMS left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this and independently validated the core scenarios rather than just reading the diff: built the Rust crate against the real bundled 1.0.83-4 CLI and ran both the wire-serialization unit tests and the new end-to-end tests (staging/superseding/resetting the Auto tier, and setModel omission preserving a staged tier) against the actual runtime — all passed against real runtime behavior, not mocks.

The API shape, tristate omit/set/reset semantics, experimental gating, and docs are consistent and well justified across all six SDKs. The only unaddressed review finding was that rust/src/types.rs's SetModelOptions lacked #[non_exhaustive], making the new auto_tier field source-breaking for external callers using struct literals, inconsistent with every sibling options struct in that file (including CapiSessionOptions). I added that attribute directly (commit 0c92063) and reran the affected tests, which still pass.

Approving and merging.

@SteveSandersonMS
SteveSandersonMS force-pushed the issue-18045-expose-auto-tier-switching-across-sdks branch from 0c92063 to cf38b7b Compare September 4, 2026 08:56
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review — PR #2514 (Auto Tier / setModel enhancements)

Reviewed the full PR diff and file list across all six SDKs. This PR adds:

  • A setAutoTier / SetAutoTier / set_auto_tier method for changing the Auto routing preference independent of the selected model.
  • A setModel/SetModel/set_model option to atomically stage an Auto tier alongside selecting the auto model (with a distinct "reset to provider default" case).
  • Updated docs for CapiSessionOptions.autoTier behavior on resume.

Result: consistent across all six SDKs. ✅ Verified parity for:

SDK New method Naming convention SetModelOptions fields
Node.js/TS session.setAutoTier(autoTier: AutoTier | null) camelCase ✓ autoTier?: AutoTier | null
Python session.set_auto_tier(auto_tier) snake_case ✓ auto_tier: AutoTier | None | _Unset (sentinel distinguishes "omit" vs explicit None)
Go session.SetAutoTier(ctx, *AutoTier) PascalCase ✓ AutoTier *AutoTier, ResetAutoTier bool
.NET session.SetAutoTierAsync(AutoTier?) PascalCase + Async suffix ✓ AutoTier, ResetAutoTier (mutually exclusive, validated)
Java session.setAutoTier(AutoTier) returning CompletableFuture<...> camelCase ✓ SetModelOptions.setAutoTier(...), setResetAutoTier(...)
Rust session.set_auto_tier(Option<AutoTier>) snake_case ✓ auto_tier: Option<AutoTierPreference> where AutoTierPreference is an enum (Tier(AutoTier) / Reset)

All implementations correctly:

  • Document the same semantics (pending status, session.model_change/session.auto_tier_switch_failed events, "only most recent request survives").
  • Reject/guard the conflicting-options case (explicit tier + reset simultaneously) — .NET, Go, and Java throw explicit exceptions; Rust avoids the invalid state entirely via its AutoTierPreference enum design; Node.js/Python use null vs. omitted-argument semantics to the same effect.
  • Mark the feature @experimental/Experimental consistently.
  • Include E2E tests (AutoTierE2ETests.cs, auto_tier_e2e_test.go, AutoTierIT.java, auto_tier.e2e.test.ts, test_auto_tier_e2e.py, rust/tests/e2e/auto_tier.rs) and README updates for every SDK.

Minor observation (not a blocking inconsistency): Rust models the tri-state choice ("leave alone" / "explicit tier" / "reset to default") as an AutoTierPreference enum rather than the AutoTier + ResetAutoTier pair used in .NET/Go/Java. This is arguably a stronger, more idiomatic design (makes the mutually-exclusive states unrepresentable) but is worth being aware of if a future SDK wants to standardize on one pattern — no change requested.

No inline comments needed; the PR maintains excellent feature parity across all language SDKs.

Generated by SDK Consistency Review Agent for #2514 · copilot · sonnet50 · 35.7 AIC · ⌖ 12.1 AIC · ⊞ 9.7K ·

@SteveSandersonMS
SteveSandersonMS added this pull request to the merge queue Sep 4, 2026
@SteveSandersonMS
SteveSandersonMS removed this pull request from the merge queue due to a manual request Sep 4, 2026
@SteveSandersonMS
SteveSandersonMS merged commit bba5abd into main Sep 4, 2026
320 of 400 checks passed
@SteveSandersonMS
SteveSandersonMS deleted the issue-18045-expose-auto-tier-switching-across-sdks branch September 4, 2026 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants