Skip to content

provider: copy catalog entries on take and skip the eager GitLab SDK import (about −23 ms) #242

Open

Description

@byapparov

Summary

Provider.state costs about 23 ms on every run. Most of that is two things:

  1. A structuredClone of the whole model catalog: 19 providers, 1,376 models.
  2. An eager import("@gitlab/gitlab-ai-provider"), used only to build a User-Agent string. It runs even without GitLab credentials.

Fixing both halves the cost: about −23 ms in a bare run, −17 ms with MCP.

Change

  1. gitlab loader: return { autoload: false } before the import when there's no key from OAuth, an API key or GITLAB_TOKEN. Same pattern as the Bedrock loader.
  2. Copy on take: copy built-in providers only when the state takes them.
    • ModelCatalog.shared() returns the entries uncopied; ModelCatalog.provider(id) returns a copy of one provider.
    • provider.ts gets a small providerDatabase() helper with peek (read-only), take (copy on first use, memoised), set and ids. ids keeps the catalog order.
    • Callers that mutate or alias a provider (mergeProvider, config extend, auth plugins) use take.
    • Callers that only read (the env scan, the custom loaders) use peek. CustomLoader is now typed Readonly<Info>.

The copy can't simply be dropped: mergeDeep aliases models, and four places mutate it.

Measurements

Measured on main @ e88ed7f432 with Bun 1.4.3. Each run was a fresh container, in-process time from process start to the first model request, median of 20 runs.

Setup main
(a) bare: no MCP, no skills 287 ms
(b) with a remote MCP server, skills and a git repo 482 ms
(b′) as (b), plus 100 ms per new TCP+TLS connection to the MCP host 678 ms
Setup Saving
(a) −23 ms
(b) −17 ms

aictrl models output is byte-identical across 8 provider and config scenarios.

Risks

  • gitlab edge case: a third-party auth plugin for gitlab with a wellknown credential and no token would no longer get the loader's options. No such plugin exists in-tree.
  • Read-only contract: loaders now see shared entries. A future loader that mutates its input would corrupt the catalog. The isolation test catches it.

Tests

  • gitlab-duo.test.ts: the SDK isn't imported without credentials (fails on main), and is imported with GITLAB_TOKEN.
  • catalog.test.ts: after a full state load with config, env, the Codex plugin and list() mutation, shared() still deep-equals catalog.json. Mutation-checked.
  • provider.test.ts: list() order and the defaultModel() choice are unchanged.
  • bun test test/provider: 347 pass.
Prototype patch
diff --git a/packages/cli/src/provider/catalog.ts b/packages/cli/src/provider/catalog.ts
index 41dd0cf6ec..b14e5b1198 100644
--- a/packages/cli/src/provider/catalog.ts
+++ b/packages/cli/src/provider/catalog.ts
@@ -18,10 +18,26 @@ export namespace ModelCatalog {
   }
 
   const entries = catalog as unknown as Record<string, Entry>
+  const builtIn: Record<string, Provider.Info> = mapValues(entries, ({ npm: _npm, api: _api, ...info }) => info)
+
+  /**
+   * The built-in providers themselves, shared by every caller. Read them without changing
+   * them; copy a provider with `provider` before changing it.
+   */
+  export function shared(): Readonly<Record<string, Provider.Info>> {
+    return builtIn
+  }
+
+  /** Returns a fresh copy of one built-in provider, because provider state mutates the providers it loads. */
+  export function provider(providerID: string): Provider.Info | undefined {
+    const info = builtIn[providerID]
+    if (!info) return undefined
+    return structuredClone(info)
+  }
 
   /** Returns fresh copies, because provider state mutates the providers it loads. */
   export function providers(): Record<string, Provider.Info> {
-    return mapValues(entries, ({ npm: _npm, api: _api, ...info }) => structuredClone(info))
+    return mapValues(builtIn, (info) => structuredClone(info))
   }
 
   export function sdkDefaults(providerID: string): Partial<SDKDefaults> {
diff --git a/packages/cli/src/provider/provider.ts b/packages/cli/src/provider/provider.ts
index a05446f568..cf0ea8f964 100644
--- a/packages/cli/src/provider/provider.ts
+++ b/packages/cli/src/provider/provider.ts
@@ -108,7 +108,8 @@ export namespace Provider {
   }
 
   type CustomModelLoader = (sdk: any, modelID: string, options?: Record<string, any>) => Promise<any>
-  type CustomLoader = (provider: Info) => Promise<{
+  /** Receives the provider without a copy, possibly the shared catalog entry: read it, never change it. */
+  type CustomLoader = (provider: Readonly<Info>) => Promise<{
     autoload: boolean
     getModel?: CustomModelLoader
     options?: Record<string, any>
@@ -443,6 +444,9 @@ export namespace Provider {
         if (auth?.type === "api") return auth.key
         return Env.get("GITLAB_TOKEN")
       })()
+      // Without a key GitLab does not autoload and its options go unused, so skip importing
+      // its SDK, which bundles several API clients.
+      if (!apiKey) return { autoload: false }
 
       const config = await Config.get()
       const providerConfig = config.provider?.["gitlab"]
@@ -677,10 +681,36 @@ export namespace Provider {
     return error
   }
 
+  /**
+   * The providers one state works with: the built-in catalog plus providers declared in config.
+   * Loading a provider mutates its provider and model objects, and the catalog is shared, so a
+   * built-in provider is copied the first time this state takes it. Providers that are only
+   * inspected, such as those without credentials, are never copied.
+   */
+  function providerDatabase() {
+    const builtIn = ModelCatalog.shared()
+    const owned: Record<string, Info> = {}
+    return {
+      /** Built-in providers in catalog order, then providers only config declares. */
+      ids: () => Object.keys({ ...builtIn, ...owned }),
+      /** Read-only: the provider this state would load, without copying it. */
+      peek: (providerID: string): Info | undefined => owned[providerID] ?? builtIn[providerID],
+      /** The provider as this state's own copy, safe to mutate. */
+      take(providerID: string): Info | undefined {
+        const provider = owned[providerID] ?? ModelCatalog.provider(providerID)
+        if (provider) owned[providerID] = provider
+        return provider
+      },
+      set(providerID: string, provider: Info) {
+        owned[providerID] = provider
+      },
+    }
+  }
+
   const state = Instance.state(async () => {
     using _ = log.time("state")
     const config = await Config.get()
-    const database = ModelCatalog.providers()
+    const database = providerDatabase()
 
     const disabled = new Set(config.disabled_providers ?? [])
     const enabled = config.enabled_providers ? new Set(config.enabled_providers) : null
@@ -710,7 +740,7 @@ export namespace Provider {
         providers[providerID] = mergeDeep(existing, provider)
         return
       }
-      const match = database[providerID]
+      const match = database.take(providerID)
       if (!match) return
       // @ts-expect-error
       providers[providerID] = mergeDeep(match, provider)
@@ -718,7 +748,7 @@ export namespace Provider {
 
     // extend database from config
     for (const [providerID, provider] of configProviders) {
-      const existing = database[providerID]
+      const existing = database.take(providerID)
       const sdkDefaults = ModelCatalog.sdkDefaults(providerID)
       const parsed: Info = {
         id: providerID,
@@ -797,13 +827,14 @@ export namespace Provider {
         )
         parsed.models[modelID] = parsedModel
       }
-      database[providerID] = parsed
+      database.set(providerID, parsed)
     }
 
     // load env
     const env = Env.all()
-    for (const [providerID, provider] of Object.entries(database)) {
+    for (const providerID of database.ids()) {
       if (disabled.has(providerID)) continue
+      const provider = database.peek(providerID)!
       const apiKey = provider.env.map((item) => env[item]).find(Boolean)
       if (!apiKey) continue
       mergeProvider(providerID, {
@@ -828,10 +859,11 @@ export namespace Provider {
       const providerID = plugin.auth.provider
       if (disabled.has(providerID)) continue
       if (!plugin.auth.loader) continue
-      const provider = database[providerID]
-      if (!provider) continue
+      if (!database.peek(providerID)) continue
       if (!(await Auth.get(providerID))) continue
 
+      // Auth plugins may change the provider's models, so they get this state's own copy.
+      const provider = database.take(providerID)!
       const options = await plugin.auth.loader(() => Auth.get(providerID) as any, provider)
       const opts = options ?? {}
       const patch: Partial<Info> = providers[providerID] ? { options: opts } : { source: "custom", options: opts }
@@ -840,7 +872,7 @@ export namespace Provider {
 
     for (const [providerID, fn] of Object.entries(CUSTOM_LOADERS)) {
       if (disabled.has(providerID)) continue
-      const data = database[providerID]
+      const data = database.peek(providerID)
       if (!data) {
         log.error("Provider does not exist in the model catalog " + providerID)
         continue
diff --git a/packages/cli/test/provider/catalog.test.ts b/packages/cli/test/provider/catalog.test.ts
index 1507107eef..e984fe272f 100644
--- a/packages/cli/test/provider/catalog.test.ts
+++ b/packages/cli/test/provider/catalog.test.ts
@@ -248,3 +248,76 @@ describe("provider registry", () => {
     })
   })
 })
+
+describe("provider state and the shared catalog", () => {
+  const catalogPath = path.join(import.meta.dir, "../../src/provider/catalog.json")
+
+  /** The catalog as it ships, read from disk so a mutated in-memory copy cannot hide a leak. */
+  async function shippedProviders(): Promise<Record<string, Provider.Info>> {
+    const entries = JSON.parse(await Bun.file(catalogPath).text()) as Record<string, Provider.Info & { npm: string }>
+    return Object.fromEntries(
+      Object.entries(entries).map(([providerID, { npm: _npm, api: _api, ...info }]: [string, any]) => [
+        providerID,
+        info,
+      ]),
+    )
+  }
+
+  test("provider returns a copy of one built-in provider", () => {
+    const copy = ModelCatalog.provider("anthropic")!
+    expect(copy).toEqual(ModelCatalog.shared()["anthropic"])
+    delete copy.models["claude-haiku-4-5"]
+    expect(ModelCatalog.shared()["anthropic"].models["claude-haiku-4-5"]).toBeDefined()
+    expect(ModelCatalog.provider("not-built-in")).toBeUndefined()
+  })
+
+  test("loading, filtering, extending and plugin changes never reach the shared catalog", async () => {
+    await using tmp = await tmpdir({
+      init: async (dir) => {
+        await Bun.write(
+          path.join(dir, "aictrl.json"),
+          JSON.stringify({
+            $schema: "https://aictrl.ai/config.json",
+            provider: {
+              // Config extends a built-in provider's models map and filters it.
+              anthropic: {
+                blacklist: ["claude-haiku-4-5"],
+                options: { timeout: 1 },
+                models: { "config-only-model": { name: "Config only" } },
+              },
+            },
+          }),
+        )
+      },
+    })
+    // The Codex auth plugin deletes OpenAI models and rewrites their costs on the provider it is given.
+    const authPath = path.join(Global.Path.data, "auth.json")
+    const savedAuth = await Bun.file(authPath)
+      .text()
+      .catch(() => "{}")
+    await Bun.write(
+      authPath,
+      JSON.stringify({ openai: { type: "oauth", access: "a", refresh: "r", expires: Date.now() + 3600000 } }),
+    )
+    try {
+      await Instance.provide({
+        directory: tmp.path,
+        init: async () => {
+          Env.set("ANTHROPIC_API_KEY", "key")
+        },
+        fn: async () => {
+          const providers = await Provider.list()
+          expect(providers["anthropic"].models["claude-haiku-4-5"]).toBeUndefined()
+          expect(providers["anthropic"].models["config-only-model"]).toBeDefined()
+          expect(providers["openai"].models["gpt-5-nano"]).toBeUndefined()
+          expect(providers["openai"].models["gpt-6.1-sol"].cost.input).toBe(0)
+          // Callers of list() may change what it returns as well.
+          providers["anthropic"].name = "changed"
+        },
+      })
+    } finally {
+      await Bun.write(authPath, savedAuth)
+    }
+    expect(ModelCatalog.shared()).toEqual(await shippedProviders())
+  })
+})
diff --git a/packages/cli/test/provider/gitlab-duo.test.ts b/packages/cli/test/provider/gitlab-duo.test.ts
index 4315016dac..5f744c59ac 100644
--- a/packages/cli/test/provider/gitlab-duo.test.ts
+++ b/packages/cli/test/provider/gitlab-duo.test.ts
@@ -1,4 +1,4 @@
-import { test, expect } from "bun:test"
+import { describe, test, expect } from "bun:test"
 import path from "path"
 
 import { tmpdir } from "../fixture/fixture"
@@ -260,3 +260,48 @@ test("GitLab Duo: has multiple agentic chat models available", async () => {
     },
   })
 })
+
+describe("GitLab Duo: SDK import", () => {
+  const entry = path.resolve(import.meta.dir, "../../src/index.ts")
+
+  /** Runs `aictrl models anthropic` in a fresh process and reports whether it evaluated the GitLab SDK. */
+  async function importsGitLabSDK(env: Record<string, string>) {
+    await using tmp = await tmpdir()
+    const probe = path.join(tmp.path, "probe.ts")
+    await Bun.write(
+      probe,
+      `process.on("exit", () => console.error("GITLAB_SDK_LOADED=" + Object.keys(require.cache).some((id) => id.includes("gitlab-ai-provider"))))`,
+    )
+    const proc = Bun.spawn(["bun", "--preload", probe, "--conditions=browser", entry, "models", "anthropic"], {
+      cwd: tmp.path,
+      // Config, auth, cache and state live under the fresh tmpdir, so no stored GitLab credential applies.
+      env: {
+        ...process.env,
+        GITLAB_TOKEN: "",
+        XDG_DATA_HOME: path.join(tmp.path, "data"),
+        XDG_CACHE_HOME: path.join(tmp.path, "cache"),
+        XDG_CONFIG_HOME: path.join(tmp.path, "config"),
+        XDG_STATE_HOME: path.join(tmp.path, "state"),
+        AICTRL_TEST_HOME: tmp.path,
+        AICTRL_TEST_MANAGED_CONFIG_DIR: path.join(tmp.path, "managed"),
+        ANTHROPIC_API_KEY: "test-key",
+        ...env,
+      },
+      stdout: "pipe",
+      stderr: "pipe",
+    })
+    const [stderr, exit] = await Promise.all([new Response(proc.stderr).text(), proc.exited])
+    expect(exit, stderr).toBe(0)
+    const report = stderr.match(/GITLAB_SDK_LOADED=(true|false)/)
+    expect(report, stderr).not.toBeNull()
+    return report![1] === "true"
+  }
+
+  test("is skipped when GitLab has no credentials", async () => {
+    expect(await importsGitLabSDK({})).toBe(false)
+  }, 30_000)
+
+  test("happens when GitLab has a token", async () => {
+    expect(await importsGitLabSDK({ GITLAB_TOKEN: "test-gitlab-token" })).toBe(true)
+  }, 30_000)
+})
diff --git a/packages/cli/test/provider/provider.test.ts b/packages/cli/test/provider/provider.test.ts
index 70eda32429..e0648a1091 100644
--- a/packages/cli/test/provider/provider.test.ts
+++ b/packages/cli/test/provider/provider.test.ts
@@ -1104,6 +1104,62 @@ test("defaultModel keeps preferring Claude Sonnet 4 for Anthropic", async () =>
   })
 })
 
+test("list keeps catalog order, then config-only providers", async () => {
+  await using tmp = await tmpdir({
+    init: async (dir) => {
+      await Bun.write(
+        path.join(dir, "aictrl.json"),
+        JSON.stringify({
+          $schema: "https://aictrl.ai/config.json",
+          provider: {
+            "config-only": {
+              npm: "@ai-sdk/openai-compatible",
+              api: "https://example.test/v1",
+              env: ["CONFIG_ONLY_API_KEY"],
+              models: { "config-model": { name: "Config model" } },
+            },
+          },
+        }),
+      )
+    },
+  })
+  await Instance.provide({
+    directory: tmp.path,
+    init: async () => {
+      Env.set("OPENAI_API_KEY", "test-api-key")
+      Env.set("ANTHROPIC_API_KEY", "test-api-key")
+      Env.set("CONFIG_ONLY_API_KEY", "test-api-key")
+    },
+    fn: async () => {
+      const loaded = Object.keys(await Provider.list())
+      const expected = ["anthropic", "openai", "config-only"]
+      expect(loaded.filter((providerID) => expected.includes(providerID))).toEqual(expected)
+    },
+  })
+})
+
+test("defaultModel takes the first loaded provider in catalog order", async () => {
+  // Other tests leave AWS credentials in process.env, so allow only the two providers under test.
+  await using tmp = await tmpdir({
+    init: async (dir) => {
+      await Bun.write(
+        path.join(dir, "aictrl.json"),
+        JSON.stringify({ $schema: "https://aictrl.ai/config.json", enabled_providers: ["openai", "anthropic"] }),
+      )
+    },
+  })
+  await Instance.provide({
+    directory: tmp.path,
+    init: async () => {
+      Env.set("OPENAI_API_KEY", "test-api-key")
+      Env.set("ANTHROPIC_API_KEY", "test-api-key")
+    },
+    fn: async () => {
+      expect(await Provider.defaultModel()).toEqual({ providerID: "anthropic", modelID: "claude-sonnet-4-6" })
+    },
+  })
+})
+
 test("multiple providers can be configured simultaneously", async () => {
   await using tmp = await tmpdir({
     init: async (dir) => {

Activity

  1. added
    enhancementNew feature or request
    P3Nice to have - Low
    performancePerformance optimization
    P4Backlog - Very low
    and removed
    P3Nice to have - Low
    on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P4Backlog - Very lowenhancementNew feature or requestperformancePerformance optimization

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions