chore(drift-sync): needs-human model-family decision (2026-07-24)

[copilotkit-devops-bot]

Needs a human decision (drift-sync)

The deterministic, zero-LLM drift-sync found a model-family change it must
NOT auto-apply (a genuinely new/unclassified family, a still-referenced
deprecation, or a registry structural mismatch). It wrote the note file(s)
below and opened this PR so the decision is REACHABLE in the repo.

Not auto-merged — a human decides.

⚠️ The Decision: include instructions in this template do NOT apply to this
PR. That protocol is scoped to a new-family note only: renderProposalNote
emits the ## Decision block solely under kind === "new-family", and
parseProposalDecision has exactly one call site — the addition half of
runDriftSyncCore. The deprecation half never reads a decision at all.

Both notes in this PR are still-referenced deprecations, and they contain
no Decision: line whatsoever. Writing one would be a no-op. There is no
marker, of any value, that makes a later run mechanically resolve this note
class — only a human registry edit can. That edit is now part of this PR
(see Human decision at the bottom), so there is no two-run hand-off here:
merging this PR both records the decision and resolves the drift.

(For reference, the original template text: for a new-family note you would
set Decision: include to classify it, or delete the note / close the PR to
reject, then merge — and the next run would apply the registry edit itself.)

Note file(s) in this PR

• drift-proposals/gemini-gemini-1.5-flash-deprecated-referenced.md

• drift-proposals/gemini-gemini-1.5-pro-deprecated-referenced.md

drift-sync outcome

npm warn Unknown project config "minimum-release-age". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown project config "block-exotic-subdeps". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
needs-human note(s) written this run — routed to human without a live re-collect (no registry edit to re-verify)
  [needs-human-still-referenced] gemini/gemini-1.5-flash: "gemini-1.5-flash" is deprecated but still referenced in source — routed to human (drift-proposals/gemini-gemini-1.5-flash-deprecated-referenced.md)
  [needs-human-still-referenced] gemini/gemini-1.5-pro: "gemini-1.5-pro" is deprecated but still referenced in source — routed to human (drift-proposals/gemini-gemini-1.5-pro-deprecated-referenced.md)
  [skipped] anthropic: live /models listing too short to trust for anthropic (10 raw id(s), need >= 19 — the number of families aimock mocks for this provider) — never mass-removing off a truncated or empty listing
npm warn Unknown env config "block-exotic-subdeps". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown env config "minimum-release-age". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown project config "minimum-release-age". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown project config "block-exotic-subdeps". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
^[[33m[STARTED]^[[39m Backing up original state...
^[[32m[COMPLETED]^[[39m Backed up original state in git stash (c62c488)
^[[33m[STARTED]^[[39m Running tasks for staged files...
^[[33m[STARTED]^[[39m package.json^[[0;90m — 2 files^[[0m
^[[33m[STARTED]^[[39m *.{ts,mts,js,mjs,cjs,json,html,css,md}^[[0;90m — 2 files^[[0m
^[[33m[STARTED]^[[39m *.{ts,mts,js,mjs}^[[0;90m — 0 files^[[0m
^[[33m[SKIPPED]^[[39m *.{ts,mts,js,mjs}^[[0;90m — no files^[[0m
^[[33m[STARTED]^[[39m prettier --write
^[[32m[COMPLETED]^[[39m prettier --write
^[[32m[COMPLETED]^[[39m *.{ts,mts,js,mjs,cjs,json,html,css,md}^[[0;90m — 2 files^[[0m
^[[32m[COMPLETED]^[[39m package.json^[[0;90m — 2 files^[[0m
^[[32m[COMPLETED]^[[39m Running tasks for staged files...
^[[33m[STARTED]^[[39m Applying modifications from tasks...
^[[32m[COMPLETED]^[[39m Applying modifications from tasks...
^[[33m[STARTED]^[[39m Cleaning up temporary files...
^[[32m[COMPLETED]^[[39m Cleaning up temporary files...
[fix/drift-2026-07-24-30074183288 85fc683] fix(drift-sync): mechanical model-family sync (needs-human note file(s))
 2 files changed, 14 insertions(+)
 create mode 100644 drift-proposals/gemini-gemini-1.5-flash-deprecated-referenced.md
 create mode 100644 drift-proposals/gemini-gemini-1.5-pro-deprecated-referenced.md
reason=needs-human
changeset-key=e08d8fcc953752d5

---

Human decision: keep mocking these families

Decision: KEEP. gemini-1.5-pro and gemini-1.5-flash are gone from Gemini's
live /models listing, but aimock still mocks them on purpose — clients pinned to
older SDKs and older CopilotKit configs still send those ids, and
isReasoningModel() must keep answering false for them.

So nothing is being removed. Instead, commit a3dc250 records the decision in the
registry by moving both families from includeFamilies.gemini into
excludeFamilies.gemini's existing "Retired / legacy specialty" bucket, and
re-pins the two affected DATA_FROZEN membership hashes.

excludeFamilies means "not counted as text-generation drift", not
"not mocked" — gemini-pro, gpt-3.5 and gemini-2.0-flash-exp already live
there while being mocked and heavily referenced in src/. Prior art for exactly
this move: #315 (fix(drift): classify bare chat-latest alias in openai excludeFamilies).

Because detectDeprecatedFamiliesForSync draws its candidate set from
includeFamilies[provider], the two families can no longer become deprecation
candidates — drift-sync stops re-flagging them needs-human, which ends the daily
red job and the daily Slack page. The two note files stay in this PR as the audit
trail for the excludeFamilies entries; they become inert on merge and will never
be re-read or re-written.

Nothing that references gemini-1.5-* was touched. src/model-utils.ts:54
(NONREASONING_FAMILIES), the three gemini reasoning-capability suites, the
gemini.test.ts / vertex-ai.test.ts conformance assertions and the recorded
sdk-shapes.ts fixture are all unchanged.

Diff

 src/__tests__/drift/logic-pin.test.ts |  4 ++--
 src/__tests__/drift/model-registry.ts | 12 +++++++++---
 2 files changed, 11 insertions(+), 5 deletions(-)

Known side effect

Gemini's truncated-listing trust floor is includeFamilies[provider].size, so it
drops from 11 to 9. Reversible: move the families back and re-pin.

---

Local red-green proof

1. Baseline — before any edit, drift suites all green

$ pnpm exec vitest run src/__tests__/drift/
 ✓ src/__tests__/drift/model-registry.ts ... (all 9 files)
 Test Files  9 passed (9)
      Tests  87 passed (87)

2. RED — registry membership moved, pins NOT yet updated

logic-pin.test.ts fails loudly on the stale membership pins, exactly as its
module doc promises:

$ pnpm exec vitest run src/__tests__/drift/logic-pin.test.ts

 × classification-data membership freeze > freezes includeFamilies.gemini membership
   → Frozen data set "includeFamilies.gemini" membership changed (now:
     ["gemini-2.0-flash","gemini-2.0-flash-lite","gemini-2.5-flash",
      "gemini-2.5-flash-lite","gemini-2.5-pro","gemini-3.1-flash-lite",
      "gemini-3.5-flash","gemini-3.5-flash-lite","gemini-3.6-flash"]).
     Expected: "4a9428b64ffcff0fbb79878d88ed993ffac53e43d64e26bcf5d86509626f593d"
     Received: "c2e2c56b8f8d5fc56152b4633e7d3782e95b7eeb9bc123da71f00e884a54a743"

 × classification-data membership freeze > freezes excludeFamilies.gemini membership
   → Frozen data set "excludeFamilies.gemini" membership changed (now:
     ["aqa","gemini-1.5-flash","gemini-1.5-pro","gemini-2.0-flash-exp", ...]).
     Expected: "e3545138234ad782937f66760bef942fca7d4bd0934a87da30bf6e5816ba69b1"
     Received: "c95dedab7588212bbba0bf9ab6434bfd43a449adbe7b35f81f48271ee849a9c2"

 Test Files  1 failed (1)
      Tests  2 failed | 18 passed (20)

The two new pin values were derived independently — sha256(JSON.stringify([...set].sort()))
recomputed from model-registry.ts in a standalone script, matching the values the
test itself reports:

includeFamilies.gemini c2e2c56b8f8d5fc56152b4633e7d3782e95b7eeb9bc123da71f00e884a54a743
excludeFamilies.gemini c95dedab7588212bbba0bf9ab6434bfd43a449adbe7b35f81f48271ee849a9c2

3. GREEN — pins re-pinned, same command

$ pnpm exec vitest run src/__tests__/drift/logic-pin.test.ts
 ✓ src/__tests__/drift/logic-pin.test.ts (20 tests) 3ms
 Test Files  1 passed (1)
      Tests  20 passed (20)

4. Full suite — zero regressions

$ pnpm test
 Test Files  165 passed | 1 skipped (166)
      Tests  4724 passed | 47 skipped (4771)

And the suites that specifically pin gemini-1.5-* behavior:

$ pnpm exec vitest run src/__tests__/model-utils.test.ts \
    src/__tests__/gemini-reasoning-capability.test.ts \
    src/__tests__/gemini-audio-reasoning-capability.test.ts \
    src/__tests__/gemini-toolonly-reasoning-capability.test.ts \
    src/__tests__/gemini.test.ts src/__tests__/vertex-ai.test.ts \
    src/__tests__/drift/ src/__tests__/drift-sync-core.test.ts
 Test Files  16 passed (16)
      Tests  236 passed (236)

5. Drift resolution proved offline

scripts/drift-sync.ts has no --dry-run / --check flag, so the canary was
exercised directly instead: detectDeprecatedFamiliesForSync is exported and
pure, so it was called against a simulated live /models listing containing every
gemini family aimock classifies except gemini-1.5-pro / gemini-1.5-flash
(31 ids — comfortably over the trust floor under both the old size 11 and the new
size 9), once on the pre-fix tree and once on the fixed tree.

Pre-fix (this branch at 85fc683) — reproduces the needs-human routing:

includeFamilies.gemini.size (trust floor) = 11
simulated live listing length            = 31
status = checked
candidates = [{"provider":"gemini","family":"gemini-1.5-flash","stillReferenced":true},
              {"provider":"gemini","family":"gemini-1.5-pro","stillReferenced":true}]
gemini-1.5 candidates    = 2
needs-human candidates   = 2
>>> STILL FLAGGED (red)

Post-fix (a3dc250) — resolved:

includeFamilies.gemini.size (trust floor) = 9
simulated live listing length            = 31
status = checked
candidates = []
gemini-1.5 candidates    = 0
needs-human candidates   = 0
>>> RESOLVED: gemini-1.5 NOT flagged

detectDeprecatedFamiliesForSync is the exact function drift-sync.ts uses to build
the deprecation candidate set, so this is the real routing surface, not a stand-in.

Plus a genuine LIVE confirmation from this PR's own CI. The drift-live-pr check
ran the drift suite against the real provider /models listings on this head commit
(preflight: Gemini key OK) and the uploaded head drift report contains exactly one
entry — a pre-existing anthropic claude-opus-5 unclassified-family finding, also
present in the base report — with zero gemini-1.5 mentions and no gemini entry
at all:

Drift gate PASSED: 1 advisory (pre-existing) finding(s) surfaced, none new-in-head.

That confirms against the live listing that moving both families to
excludeFamilies introduced no new drift and no unclassified-family regression
(isClassifiedFamily is satisfied by include or exclude). To be precise about
what it does not prove: this leg does not surface the still-referenced-deprecation
class as a report entry — the base report shows zero gemini-1.5 mentions too — so
the live job is regression evidence, and the needs-human routing fix itself is
proved by the detectDeprecatedFamiliesForSync red-green above.

6. Pre-push gates

$ pnpm format:check   → All matched files use Prettier code style!
$ pnpm lint           → clean (no output)
$ pnpm exec tsc --noEmit → clean (no output)
$ pnpm build          → ✔ Build complete (273 files)
$ pnpm test:exports   → No problems found 🌟

7. Zero behavior change → no version bump, no changeset

The test that matters is not "which directory did the files sit in" but does this
change aimock's behavior for a consumer? It does not, and that is verified three
independent ways.

(a) The registry is not reachable from shipped code. Every importer of
src/__tests__/drift/model-registry.ts is a test or the maintenance script:

scripts/drift-sync.ts                              (repo script, not published)
src/__tests__/drift-sync-core.test.ts
src/__tests__/drift-sync-mirror-equivalence.test.ts
src/__tests__/drift/logic-pin.test.ts
src/__tests__/drift/model-registry.test.ts
src/__tests__/drift/models.drift.ts

No shipped-source importer exists — src/model-utils.ts, src/server.ts, the
handlers and every published entrypoint in tsdown.config.ts are all absent from
that list. includeFamilies / excludeFamilies are drift-detection metadata, not
a runtime allowlist. Confirmed at the artifact level too: dist/ contains zero
occurrences of includeFamilies, excludeFamilies or isClassifiedFamily, so the
data isn't inlined either, and npm pack --dry-run ships 576 files with none
matching __tests__ / /drift/ / model-registry / logic-pin / src/
(files allowlist is ["dist","fixtures",".claude-plugin","skills","CHANGELOG.md"];
scripts/ ships 0 entries; no .npmignore).

(b) The published artifact is byte-identical. Built dist/ from the pre-fix and
post-fix trees and compared all 285 shipped code/declaration files. Note the build
is not reproducible — two builds of identical source differ in declaration
sourcemaps and in tsdown's non-deterministic import-alias numbering
(node_http0 ↔ node_http1), so a naive whole-dist hash gives a false positive.
Normalizing that noise, with two identical-source builds as the control noise floor:

control: two IDENTICAL-source builds   → files=285 differing=0
subject: PRE-fix vs POST-fix build     → files=285 differing=0
VERDICT: PUBLISHED ARTIFACT SEMANTICALLY IDENTICAL

(c) Consumer-visible behavior is bit-identical. Probed the built dist/ before
and after with real requests for both ids — generateContent, SSE
streamGenerateContent, strict-on and strict-off reasoning, the OpenAI-compatible
/v1/chat/completions surface, and /v1/models — plus the isReasoningModel()
verdicts and the captured capability warnings. The two JSON captures compare equal:

PRE  isReasoningModel: {'gemini-1.5-pro': False, 'gemini-1.5-flash': False}
POST isReasoningModel: {'gemini-1.5-pro': False, 'gemini-1.5-flash': False}
controls (unchanged): gemini-2.5-pro=True, gemini-pro=True, some-future-model=True
8/8 request probes identical (all HTTP 200, identical bodies)
capability warnings identical, still naming both gemini-1.5 ids
identical overall: True

So aimock still mocks both ids identically, isReasoningModel() still returns
false for both, and the advertised model list is unchanged.

Release mechanism. publish-release.yml triggers on push to main, compares
package.json's version against npm, and publishes + tags only when that version is
not yet published — i.e. releases come from a manual package.json version bump.
There is no Changesets setup (no .changeset/, no @changesets/cli), no
release-please and no semantic-release, and no workflow derives a version from commit
messages, so the conventional-commit type does not trigger a release; the
fix(drift): prefix here matches the repo's own prior art for registry
classification changes (#315, #322) and is inert with respect to versioning.
publish-commit.yml → pkg-pr-new is just the per-PR preview publish behind the
"Continuous Releases" / "preview" checks and gates nothing on versioning. No
changeset file and no version bump are required.

Minor pre-existing caveat: unreleased-check.yml counts commits touching src/
since the last tag without excluding src/__tests__/, so on merge it will emit a
::warning:: that the version wasn't bumped. It is a warning only — no exit 1 —
so it does not fail CI. #336 had the same shape.

---

Separate latent defect, filed for follow-up (NOT introduced by this PR)

The mechanical registry-edit path documented for new-family notes is currently
dead-locked by its own gate, so ok-applied can never happen: gate-2 of
drift-sync-check.ts re-runs logic-pin.test.ts as a subprocess, that file's
DATA_FROZEN block freezes set membership, and a mechanical add/remove is a
membership change — so it trips the pin, gets reverted by revertFiles, and
reports gate-failed. #335 shipped the mechanical path; #336 added DATA_FROZEN
40 minutes later. It has never been observed because no ok-applied run has ever
occurred. The ok-applied test injects a fake gate (makeFakeDeps), so it asserts
runSyncCheck was called, never that the real pin would pass. Worth fixing
separately — either exempt the DATA_FROZEN membership hashes from gate-2, or have
drift-sync re-pin the hash as part of the allowlisted data edit and lean on human
PR review as the guard.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@337

commit: a3dc250

[jpr5]

Human decision recorded: KEEP mocking gemini-1.5-pro / gemini-1.5-flash.

Both are gone from Gemini's live /models listing but remain legitimately referenced in aimock (notably NONREASONING_FAMILIES in src/model-utils.ts, plus the capability suites that use them as the non-reasoning exemplar). Users pinned to older SDKs still hit these ids, so removing the mocks would be destructive. Classifying them under excludeFamilies is the established pattern here (gemini-pro, gpt-3.5, gemini-2.0-flash-exp; prior art #315/#322).

Verified before approving:

• Zero behavior change, three independent ways: no shipped-source importer of model-registry.ts (only 5 tests + scripts/drift-sync.ts); pre/post dist semantically identical across all 285 shipped files; built-artifact probe over both ids (generateContent, SSE, strict on/off, /v1/chat/completions, /v1/models, isReasoningModel) compares equal.
• Genuine red-green: registry edit alone fails the two membership pins, re-pin turns it green on the same command. Pins independently derived, three-way agreement with vitest's Received.
• Full suite 4724 passed / 0 failed; 236 passed across the gemini-1.5 / isReasoningModel suites.
• Drift actually resolves: drove the real detectDeprecatedFamiliesForSync (no --dry-run exists) — 2 needs-human candidates pre-fix, 0 post-fix. CI's drift-live-pr agrees against the live listing.
• No changeset or version bump: releases are manual package.json bumps via publish-release.yml, and nothing derives a version from commit messages.

Commit is exactly the 2 intended test files, +11/-5.

[jpr5]

Human decision: KEEP mocking gemini-1.5-pro / gemini-1.5-flash. Verified zero behavior change (no shipped importer of model-registry.ts; pre/post dist semantically identical across 285 shipped files; built-artifact probe over both ids compares equal), genuine red-green on the membership pins, full suite 4724 passed / 0 failed, drift resolves 2 candidates to 0 via the real detector, and no changeset needed (manual version bumps, nothing derives versions from commit messages).