Skip to content

feat: Ollama live drift leg via CI-provisioned daemon - #326

Merged
jpr5 merged 2 commits into
mainfrom
feat/ollama-live-drift
Jul 23, 2026
Merged

feat: Ollama live drift leg via CI-provisioned daemon#326
jpr5 merged 2 commits into
mainfrom
feat/ollama-live-drift

Conversation

@jpr5

@jpr5 jpr5 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What

Brings the Ollama drift surface into live CI coverage. Today src/__tests__/drift/ollama.drift.ts has a live leg gated on OLLAMA_HOST, but no Ollama daemon exists in CI and there is no static fallback, so the leg is silently skipped — zero CI signal. Ollama needs no API key; the only "credential" is a running local daemon, so we can provision one in CI for free.

Changes

  • test-drift.yml (scheduled/dispatch drift job) and fix-drift.yml (fix job): added an additive Provision Ollama daemon step that installs Ollama, starts ollama serve in the background, waits for readiness, and pulls the tiny qwen2:0.5b (~350 MB, smallest model supporting both /api/chat and /api/generate). Exported OLLAMA_HOST=127.0.0.1:11434 + OLLAMA_MODEL=qwen2:0.5b into the collector/verify steps so the leg runs against the real daemon. fix-drift.yml's post-fix verify + authoritative re-collect steps are also un-gated so the auto-fix path can actually verify an Ollama fix (daemon is already running in that job).
  • ollama.drift.ts: model is now OLLAMA_MODEL ?? "llama3.2" (lets CI use the tiny model while a local dev run defaults unchanged); OLLAMA_HOST is normalized to accept both a full base URL and Ollama's native bare host:port form. No assertion logic changed.

⚠️ CI-time cost (for merge consideration)

The daemon install + model pull adds ~2-4 min to every drift run (daily scheduled test-drift + every fix-drift invocation). This is the tradeoff for real Ollama coverage. Weigh at merge — if too costly, options are caching the model or gating the leg to a less frequent cadence.

Red-green proof (LIVE, real Ollama daemon)

Ran locally against a real ollama serve + qwen2:0.5b (no key needed).

GREEN (baseline, unmodified mock):

OLLAMA_HOST=127.0.0.1:11434 OLLAMA_MODEL=qwen2:0.5b npx vitest run src/__tests__/drift/ollama.drift.ts --config vitest.config.drift.ts
 ✓ src/__tests__/drift/ollama.drift.ts (3 tests) 1256ms
 Test Files  1 passed (1)
      Tests  3 passed (3)

RED (temporarily dropped done_reason from the mock's non-streaming /api/chat builder — a genuine mock/real divergence):

 × Ollama drift > /api/chat response shape matches
API DRIFT DETECTED: Ollama /api/chat
  Surface: ollama

  1. [critical] LLMOCK DRIFT — field in SDK + real API but missing from mock
     Path:    done_reason
     SDK:     string
     Real:    string
     Mock:    <absent>

The Surface: ollama marker is emitted → the collector maps it to exit 2 (auto-fixable), confirming the leg has teeth against the real API.

GREEN again (reverted): 3/3 pass; working tree clean (only the test + workflows changed).

Skip-safe: with no OLLAMA_HOST (local dev / non-CI), the leg still cleanly skips (3 skipped) — no regression.

Other checks

  • Tests-inclusive ES2022 typecheck on the touched test file: 0 new errors (pre-existing errors in other test files unchanged).
  • prettier ✓, eslint ✓ (exit 0), pnpm build ✓, full unit suite ✓ (4684 passed).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KamK73Wu5heLxJEogM6hHC

@pkg-pr-new

pkg-pr-new Bot commented Jul 22, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@326

commit: 211ae8b

jpr5 added 2 commits July 22, 2026 17:06
@jpr5
jpr5 marked this pull request as ready for review July 23, 2026 00:06
@jpr5
jpr5 force-pushed the feat/ollama-live-drift branch from c4a81c1 to 211ae8b Compare July 23, 2026 00:06
@jpr5
jpr5 merged commit e885dd7 into main Jul 23, 2026
29 checks passed
@jpr5
jpr5 deleted the feat/ollama-live-drift branch July 23, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant