Repository navigation
Feature: Custom listModels handler for BYOK mode #729
Description
Summary
In BYOK (Bring Your Own Key) mode, client.listModels() sends a models.list RPC to the CLI server, which may not know what models the user's custom provider supports. Users need a way to provide their own model list so downstream consumers can discover available models in a compliant ModelInfo[] format.
Proposal
Add an optional onListModels handler to CopilotClientOptions across all 4 SDKs. When provided, client.listModels() calls the handler instead of querying the CLI server.
Design
- Placement: Client-level (
CopilotClientOptions), not session-level - Behavior: Completely replaces the CLI RPC call when provided (no fallback/merge)
- Caching: Handler results are cached the same way as CLI results
- Async: Handler may be sync or async (all SDKs support it)
Naming per SDK
| SDK | Field | Type |
|---|---|---|
| Node | onListModels |
() => Promise<ModelInfo[]> | ModelInfo[] |
| Python | on_list_models |
Callable[[], list[ModelInfo] | Awaitable[list[ModelInfo]]] |
| Go | OnListModels |
func(ctx context.Context) ([]ModelInfo, error) |
| .NET | OnListModels |
Func<CancellationToken, Task<List<ModelInfo>>>? |
Example (TypeScript)
const client = new CopilotClient({
onListModels: () => [
{
id: "my-custom-model",
name: "My Custom Model",
capabilities: {
supports: { vision: false, reasoningEffort: false },
limits: { max_context_window_tokens: 128000 },
},
},
],
});Motivation
When using BYOK with providers like Ollama, Azure AI Foundry, or custom OpenAI-compatible endpoints, the CLI server cannot enumerate available models. This forces callers to hardcode model names instead of discovering them programmatically. A custom handler lets users return their provider's models in the standard format.
Activity