Summary
In BYOK (Bring Your Own Key) mode, client.listModels() sends a models.list RPC to the CLI server, which may not know what models the user's custom provider supports. Users need a way to provide their own model list so downstream consumers can discover available models in a compliant ModelInfo[] format.
Proposal
Add an optional onListModels handler to CopilotClientOptions across all 4 SDKs. When provided, client.listModels() calls the handler instead of querying the CLI server.
Design
- Placement: Client-level (
CopilotClientOptions), not session-level
- Behavior: Completely replaces the CLI RPC call when provided (no fallback/merge)
- Caching: Handler results are cached the same way as CLI results
- Async: Handler may be sync or async (all SDKs support it)
Naming per SDK
| SDK |
Field |
Type |
| Node |
onListModels |
() => Promise<ModelInfo[]> | ModelInfo[] |
| Python |
on_list_models |
Callable[[], list[ModelInfo] | Awaitable[list[ModelInfo]]] |
| Go |
OnListModels |
func(ctx context.Context) ([]ModelInfo, error) |
| .NET |
OnListModels |
Func<CancellationToken, Task<List<ModelInfo>>>? |
Example (TypeScript)
const client = new CopilotClient({
onListModels: () => [
{
id: "my-custom-model",
name: "My Custom Model",
capabilities: {
supports: { vision: false, reasoningEffort: false },
limits: { max_context_window_tokens: 128000 },
},
},
],
});
Motivation
When using BYOK with providers like Ollama, Azure AI Foundry, or custom OpenAI-compatible endpoints, the CLI server cannot enumerate available models. This forces callers to hardcode model names instead of discovering them programmatically. A custom handler lets users return their provider's models in the standard format.
Summary
In BYOK (Bring Your Own Key) mode,
client.listModels()sends amodels.listRPC to the CLI server, which may not know what models the user's custom provider supports. Users need a way to provide their own model list so downstream consumers can discover available models in a compliantModelInfo[]format.Proposal
Add an optional
onListModelshandler toCopilotClientOptionsacross all 4 SDKs. When provided,client.listModels()calls the handler instead of querying the CLI server.Design
CopilotClientOptions), not session-levelNaming per SDK
onListModels() => Promise<ModelInfo[]> | ModelInfo[]on_list_modelsCallable[[], list[ModelInfo] | Awaitable[list[ModelInfo]]]OnListModelsfunc(ctx context.Context) ([]ModelInfo, error)OnListModelsFunc<CancellationToken, Task<List<ModelInfo>>>?Example (TypeScript)
Motivation
When using BYOK with providers like Ollama, Azure AI Foundry, or custom OpenAI-compatible endpoints, the CLI server cannot enumerate available models. This forces callers to hardcode model names instead of discovering them programmatically. A custom handler lets users return their provider's models in the standard format.