Tags: CopilotKit/aimock
Tags
chore: release v1.37.4 (#320) Release v1.37.4. Bumps `package.json` to publish the fix from #319 to npm + ghcr. ## What ships - **#319** — `hasToolResult` record/replay symmetry via shared `currentTurnHasToolResult` helper (recorder previously stamped whole-conversation while the matcher read current-turn, so genuinely-recorded multi-turn fixtures could never match), plus Anthropic `tool_result`+text ordering fix. Completes the incomplete #316 / v1.37.3 fix. - CHANGELOG entries for 1.37.4 and the retroactive 1.37.3 (which shipped undocumented) already landed in #319. Consumer: unblocks CopilotKit showcase multi-turn recording once bumped to 1.37.4. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
chore: release v1.37.4 (#320) Release v1.37.4. Bumps `package.json` to publish the fix from #319 to npm + ghcr. ## What ships - **#319** — `hasToolResult` record/replay symmetry via shared `currentTurnHasToolResult` helper (recorder previously stamped whole-conversation while the matcher read current-turn, so genuinely-recorded multi-turn fixtures could never match), plus Anthropic `tool_result`+text ordering fix. Completes the incomplete #316 / v1.37.3 fix. - CHANGELOG entries for 1.37.4 and the retroactive 1.37.3 (which shipped undocumented) already landed in #319. Consumer: unblocks CopilotKit showcase multi-turn recording once bumped to 1.37.4. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
chore: release v1.37.2 (#309) ## Release v1.37.2 (patch) Cuts a patch release for the journal body cap + error stack instrumentation fix merged in #308. ### Version bump `package.json` `1.37.1` → `1.37.2`. Also bumps `packages/aimock-pytest/src/aimock_pytest/_version.py` `AIMOCK_VERSION` to `1.37.2` (pins the pytest harness to the released npm version). ### CHANGELOG ## [1.37.2] - 2026-07-17 ### Fixed - Prevent `Invalid string length` crash on `GET /__aimock/journal` by capping retained request bodies at 64 KB (#308) ### Changed - Log full error stack on unhandled request errors for prod diagnosis (#308) ### Quality gate - `pnpm run format:check` — pass - `pnpm run lint` — pass - `tsc --noEmit` — pass - `pnpm test` — 4549 passed (154 files) - `pnpm run build` — pass
fix(drift): enforce real green-gate before auto-merge (pass>0, requir… …ed contexts, no false-green) (#306) ## What & why The in-workflow auto-merge "green gate" in `fix-drift.yml` did **not** actually enforce green. It counted `gh pr checks | wc -l` rows (a "no checks reported" message is itself a row) and relied on `gh pr checks --watch --fail-fast`, which **exits 0 on empty / pending / skipped / neutral**. That is a false-green: an unverified drift PR could auto-merge to `main`. This fix-forward replaces the gate with a real, testable assertion. ## Findings addressed 1. **CRITICAL — false-green gate.** Replaced row-count + `--watch`-exit-0 with a machine-readable assertion: poll on `gh pr checks --json` length for registration, `--watch` under a hard timeout, then re-query JSON and require **pass>=1 AND zero in {pending, fail, cancel}**. neutral/skipped do **not** count toward pass. 2. **CRITICAL — required contexts present.** Gate asserts the expected required contexts (`prettier`, `eslint`, `exports`, `commitlint`, `test (20/22/24)`, `agui-schema-drift`, `drift-live-pr`, `zizmor` — derived from the real check set on drift PR #305) are present **and** concluded green, so one unrelated fast check can't satisfy the gate and a registration race can't slip an incomplete set through. 3. **HIGH — wrong-PR risk.** Select the PR by head-SHA match (`headRefOid == git rev-parse HEAD`), not `gh pr list ... .[0]`. Added `set -euo pipefail` to the multi-command run blocks. 4. **HIGH — checks:read.** Added `checks: read` + `statuses: read` to the job `permissions:` and to the minted app-token permissions. 5. **HIGH — masked autofix failure.** `continue-on-error: true` autofix that fails now triggers a distinct "autofix STEP failed" Slack alert and **fails the job** (`steps.autofix.outcome != 'success'`). 6. **HIGH — exit-5 quarantine silent skip.** Collector exit 5 now sets a `quarantine=true` output and routes to a distinct "drift quarantine — needs human" Slack alert instead of the silent skip path. 7. **MED — Slack correctness/coverage.** Success message says "merged to main" only when the merge command returned 0 (`merged=true` output); otherwise "PR open, not yet merged". Missing `SLACK_WEBHOOK` on the failure path is now a visible `::error::` annotation, not a silent `exit 0`. Dropped `curl -s` for `curl -fsS --show-error`. 8. **MED — watch timeout.** `--watch` wrapped in `timeout 1200`; the failure Slack distinguishes `timeout` / `not-green` / `no-checks` reasons. ## Design The gate decision is factored into a standalone, unit-testable script `scripts/ci-merge-gate.sh` that reads the `gh pr checks --json name,state,bucket` array (file arg or stdin) and exits `0` **only** on true-green. The workflow calls this script as the authoritative decision (the `--watch` exit status is advisory only). Required contexts default to the drift-PR set and are overridable via `REQUIRED_CONTEXTS`. ## Red-green proof The gate is exercised by `src/__tests__/ci-merge-gate.test.ts` (13 tests) plus a direct shell comparison. ### RED — OLD workflow logic (row-count + `--watch`) treats false-green as mergeable ``` [a-empty] OLD gate => WOULD MERGE (rows=1, watch exit 0) <-- FALSE GREEN [b-pending] OLD gate => WOULD MERGE (rows=1, watch exit 0) <-- FALSE GREEN [c-skipped] OLD gate => WOULD MERGE (rows=2, watch exit 0) <-- FALSE GREEN [d-unrelated-pass] OLD gate => WOULD MERGE (rows=1, watch exit 0) <-- FALSE GREEN ``` ### GREEN — NEW gate (`scripts/ci-merge-gate.sh`) ``` [a-empty] NEW gate => REFUSE (exit 1) <-- correctly blocked [b-pending] NEW gate => REFUSE (exit 1) <-- correctly blocked [c-skipped] NEW gate => REFUSE (exit 1) <-- correctly blocked [d-unrelated-pass] NEW gate => REFUSE (exit 1) <-- correctly blocked [e-all-green] NEW gate => ACCEPT (exit 0) ``` ### vitest ``` ✓ src/__tests__/ci-merge-gate.test.ts (13 tests) ci-merge-gate.sh — RED: old logic treats false-green as mergeable ✓ (a) empty/no-checks — OLD gate would MERGE ✓ (b) pending-only — OLD gate would MERGE ✓ (c) skipped/neutral-only — OLD gate would MERGE ✓ (d) one unrelated pass, required missing — OLD gate would MERGE ci-merge-gate.sh — GREEN: new gate refuses false-green, accepts true-green ✓ (a) empty/no-checks — REFUSES (exit 1) ✓ (b) pending-only — REFUSES (exit 1) ✓ (c) skipped/neutral-only — REFUSES (exit 1) and does not count skips as pass ✓ (d) one unrelated pass with required missing — REFUSES (exit 1) ✓ (e) all-required-green (extras skipped) — ACCEPTS (exit 0) ✓ fails when a required context is in the fail bucket ✓ fails when a required context is still pending even if others pass ✓ honors REQUIRED_CONTEXTS override (comma-separated) ✓ derives bucket from raw state when bucket field is absent Test Files 1 passed (1) Tests 13 passed (13) ``` ## Local verification - `actionlint .github/workflows/fix-drift.yml` — clean - `shellcheck scripts/ci-merge-gate.sh` — clean - `pnpm format:check` — clean - `pnpm lint` — clean - gate re-validated against real PR #305 JSON (all-required-green) => exit 0 ## Scope Diff touches only: `.github/workflows/fix-drift.yml`, `scripts/ci-merge-gate.sh`, `src/__tests__/ci-merge-gate.test.ts`. No ruleset, secret, or GitHub App / bypass-config changes. **Opened as DRAFT — do not merge; orchestrator gates the merge.**
feat: toolResultContains match gate on the last tool-result payload (#… …299) ## Why The showcase mastra interrupt demos use a native suspend tool where pick and cancel resume with the SAME tool_call_id; the requests differ only inside the tool-result message content (`{"chosen_time": ...}` vs `{"cancelled": true}`). Every existing JSON-expressible gate is identical between the two requests, so first-match-wins replays the pick confirmation after the user cancelled. `predicate` cannot help: the JSON fixture loader has no way to express it. ## What New `match.toolResultContains: string` gate: passes when the request's LAST message is a `role: "tool"` result whose text content contains the substring. Mirrors the existing `toolCallId` last-message rule so the two stay consistent. - `src/router.ts`: gate placed with the other shape predicates, right after `toolCallId` - `src/types.ts`: `FixtureMatch` + `FixtureFileEntry.match` - `src/fixture-loader.ts`: `entryToFixture` carry-through; validation (string, non-empty — empty substring would silently act as a catch-all); userMessage dedup key; catch-all discriminator set - `packages/aimock-pytest`: `toolResultContains` added to `_MATCH_LEVEL_OPT_KEYS` (0.5.0) - docs: fixtures + multi-turn pages, README, write-fixtures skill - tests: approve/cancel discrimination sharing one toolCallId; substring present/absent; no-tool-message request never matches; stale tool result followed by a new user turn never matches; array-of-parts content; null content; loader validation + dedup + carry-through ## Version 1.37.0 (changelog entry added with this PR's number in a follow-up commit on this branch). --- ### Maintainer note — additional fixes folded in during review Beyond the `toolResultContains` gate, this branch now also carries (all under the single 1.37.0 release): - **fix:** completed the duplicate-userMessage dedup key so legitimately-distinct fixtures no longer emit spurious shadow warnings (kind-aware matcher serialization; `predicate` keyed by identity). - **fix:** guarded `recordedTimings.interChunkDelaysMs` against non-array values — a malformed entry no longer throws and silently drops every fixture in its file. - **fix:** bumped the `aimock-pytest` default server pin to 1.37.0 so the auto-download path actually supports the new kwarg the client forwards. - **ci:** the `release` script now runs `format:check` + `test:exports` before publish. - **docs:** corrected the match-field / dedup / `toolCallId` semantics docs, and fixed the write-fixtures E2E lifecycle to use `resetMatchCounts()` instead of a full `reset()` between tests. (These consolidate PR #300, now closed, into this one release. CHANGELOG updated accordingly.)
chore: release v1.36.1 (#295) ## Release v1.36.1 (patch) Cuts a patch release for the `llm` config passthrough fix merged in #294. ### Version bump `package.json` `1.36.0` → `1.36.1`. (Matches this repo's practiced release convention: only `package.json` + `CHANGELOG.md` are version-carrying for aimock — `charts/aimock/Chart.yaml`, `.claude-plugin/*`, and `src/cli.ts` are not bumped per release here, as v1.36.0 also demonstrated, and no CI parity check enforces them.) ### CHANGELOG Rolled the existing `[Unreleased]` entry into `## [1.36.1] - 2026-07-14`: > ### Fixed > - `aimock.json` now honours `llm.latency`, `llm.chunkSize`, `llm.replaySpeed` and `llm.logLevel`, which the config loader previously accepted and ignored. Each behaves as its `llmock` CLI flag and `createMockSuite({ llm })` equivalent. `llm.replaySpeed` divides every delay source (recorded timings, streaming profiles, `latency`), so a fixture suite can replay recorded inter-chunk timings faster than they were recorded; a fixture's own `replaySpeed` still takes precedence. A non-positive `llm.replaySpeed` is ignored with a warning, matching the existing fixture-level guard (#294) ### Docs completeness audit Checked every place the `llm` config is described: | Location | Before | Action | |---|---|---| | `docs/aimock-cli/index.html` — Config Fields prose row | all four fields listed (fixed in #294) | already complete | | `docs/aimock-cli/index.html` — `aimock.json` JSON example | showed `latency`, `chunkSize`, `logLevel`; **missing `replaySpeed`** | **added `"replaySpeed": 1`** so the example matches the documented field set | | `src/config-loader.ts` — `AimockConfig.llm` type | all four typed; no per-field JSDoc (existing style) | no change (would not force JSDoc where none exists) | | `docs/index.html` landing example | illustrative marketing snippet (`providers` array, stale banner), not a field reference | no change | | `docs/chat-completions/index.html`, `packages/aimock-pytest/README.md`, `README.md` | partial illustrative examples, not a full `llm` field reference | no change | Net: one real gap fixed (JSON example ↔ prose consistency on the canonical config-reference page); everywhere else was already complete or is deliberately illustrative. ### Quality gate - `pnpm run format:check` — pass (changed files clean; only a gitignored `.impeccable/hook.cache.json` local artifact warns) - `pnpm run lint` — pass - `pnpm test` — 4364 passed (147 files) - `pnpm run build` — pass Draft until you're ready to merge; merging to `main` triggers `publish-release.yml` (npm publish + tag + GitHub Release + Slack).
feat: aimock-owned upstream provider keys on fixture-miss passthrough (… …#293) ## What Opt-in, backward-compatible support for **aimock owning the upstream provider API key**. On a fixture-miss passthrough, aimock forwards the request to the real upstream. Callers commonly send a **dummy placeholder key** (e.g. `sk-aimock-…`) only to satisfy an SDK's non-empty-key requirement; today that dummy is forwarded verbatim and the upstream rejects it. This PR lets aimock inject its own configured key in that case, while never clobbering a real caller key. ## Design - **Env-sourced keys** (not CLI flags — secrets stay out of `ps`): `AIMOCK_PROVIDER_OPENAI_KEY`, `AIMOCK_PROVIDER_ANTHROPIC_KEY`, `AIMOCK_PROVIDER_GEMINI_KEY` → plumbed into new `RecordConfig.providerKeys` (parallel to `providers`), wired from `cli.ts` via `readProviderKeysFromEnv()`. - **`applyProviderAuth(forwardHeaders, target, providerKey, builtinKey)`** invoked immediately after `buildForwardHeaders(req)` on the proxy path. Provider-aware schemes: - OpenAI / OpenRouter / Cohere → `Authorization: Bearer <key>` - Anthropic → `x-api-key: <key>` - Gemini (+ gemini-interactions) → `x-goog-api-key: <key>` - **Precedence (dummy-override semantics):** caller credential absent or dummy-prefixed (`sk-aimock-`, overridable via `AIMOCK_DUMMY_KEY_MARKER`) → inject built-in key; real caller key → forwarded unchanged (caller overrides); no built-in key configured → no-op (feature inert, verbatim forward). - **Excluded:** Bedrock (SigV4) / Vertex / Azure-AD (OAuth) — signed/exchanged credentials are never rewritten; those keep forwarding caller creds. ### Gemini mechanism decision Google's Generative Language API accepts the key via either `x-goog-api-key` **header** or a `?key=` query param. This PR uses the **header** form (consistent with the header-injection approach for every other provider, and no URL mutation). The `target: URL` param to `applyProviderAuth` is reserved (currently `void`) so a query-param scheme can be added later without a signature change. ## Red-Green Proof (real forwarded-header surface) The test stands up a **fake upstream HTTP server** that records the `Authorization` / `x-api-key` / `x-goog-api-key` it receives, points aimock's proxy at it in `--proxy-only` mode, sends a request through aimock, and asserts on the header the fake upstream actually saw. **RED** — injection call disabled in `recorder.ts`: ``` × GREEN: built-in key set + caller sends dummy → aimock injects its own key → expected 'Bearer sk-aimock-dev-ci-only' to be 'Bearer sk-real-test-123' × no caller credential + built-in key set → aimock injects its own key → expected undefined to be 'Bearer sk-real-test-123' × Anthropic: built-in key set + caller sends dummy → injects x-api-key → expected 'sk-aimock-dev-ci-only' to be 'sk-real-test-123' × Gemini: built-in key set + caller sends dummy → injects x-goog-api-key → expected 'sk-aimock-dev-ci-only' to be 'sk-real-test-123' Tests 4 failed | 6 passed (10) ``` The dummy `sk-aimock-dev-ci-only` reaching the upstream verbatim proves today's forwarding behavior. **GREEN** — feature enabled: ``` ✓ GREEN: built-in key set + caller sends dummy → aimock injects its own key (fake upstream saw: Authorization: Bearer sk-real-test-123) Test Files 1 passed (1) Tests 10 passed (10) ``` ### Backward-compat cases covered (all pass) - (a) Feature OFF (no built-in key) → dummy forwarded unchanged. - (b) Caller sends a REAL key (not `sk-aimock-`) → forwarded unchanged even with a built-in key set (caller overrides). - (c) A fixture-**matched** request never triggers injection (short-circuits before `proxyAndRecord`; fake upstream request count = 0). - (d) Anthropic uses `x-api-key`; Gemini uses `x-goog-api-key`. ## Files changed - `src/provider-auth.ts` (new) — injection logic + env reader + dummy marker. - `src/recorder.ts` — invoke `applyProviderAuth` after `buildForwardHeaders`. - `src/cli.ts` — plumb `providerKeys` from env into `RecordConfig`. - `src/types.ts` — add `RecordConfig.providerKeys`. - `src/__tests__/provider-auth.test.ts` (new) — real-surface red-green suite. ## Verification `prettier --check`, `eslint`, `tsc --noEmit`, `tsdown` build, and the full `vitest` suite (**4344 tests / 147 files**) all pass. --- **DRAFT** pending finalization of the concurrent design spec.
chore: release v1.35.1 (#290) Patch release rolling up the #288 record-mode fixture fix. ## What's in this release - **Fixed (#288):** Record mode no longer silently drops fixtures when an SDK closes its socket immediately after `data: [DONE]` (e.g. the OpenAI Python SDK); the completed upstream response is now persisted. Genuine mid-stream aborts (client disconnect before `[DONE]`) still abort upstream and write no fixture. ## Release mechanics Standard version bump across `package.json`, `.claude-plugin/plugin.json`, `.claude-plugin/marketplace.json`, `charts/aimock/Chart.yaml`, and the aimock-pytest version pin, plus the CHANGELOG `1.35.1` section. Merging this triggers the `publish-release.yml` auto-publish (npm + git tag + GitHub Release + Docker image), since `package.json` is now `1.35.1` which is not yet on npm.
chore: release v1.35.0 (#287) ## Release v1.35.0 Version bump cut off latest `main` (includes #285 blocks-completion and #286 video-testid). ### Version surfaces bumped (1.34.0 → 1.35.0) These five literal build surfaces track the aimock npm release line and were bumped together: 1. `package.json` — `"version"` → `1.35.0` 2. `.claude-plugin/plugin.json` — `"version"` → `1.35.0` 3. `.claude-plugin/marketplace.json` — `plugins[0].source.version` → `^1.35.0` (caret preserved) 4. `charts/aimock/Chart.yaml` — `appVersion` → `"1.35.0"` 5. `packages/aimock-pytest/src/aimock_pytest/_version.py` — `AIMOCK_VERSION` → `"1.35.0"` ### Deliberately NOT changed (independently versioned) Verified against release history — these do **not** track the aimock release line and follow their own cadence, so they are left as-is: - `charts/aimock/Chart.yaml` `version:` stays **`0.1.0`** (Helm chart version — independent of the app it packages) - `packages/aimock-pytest/pyproject.toml` `version` stays **`0.4.0`** (the Python package's own release cadence, distinct from the npm release it pins via `AIMOCK_VERSION`) ### Hygiene fix - `packages/aimock-pytest/README.md` — corrected the stale `--aimock-version` default from `1.33.0` → `1.35.0` to match `AIMOCK_VERSION`. ### CHANGELOG - Renamed the accumulated `## [Unreleased]` section to `## [1.35.0] - 2026-06-27` and inserted a fresh empty `## [Unreleased]` above it. Bullet contents unchanged. ### What's in 1.35.0 - **`blocks` feature completed across all providers** — ordered text/tool-call block streaming now honored on replay across Anthropic, OpenAI (Responses + chat-completions), Gemini, Ollama, Cohere, Bedrock (invoke + Converse), and Gemini Interactions, with record-side block capture where the wire protocol allows (#274). - **Blocks-only fixtures are first-class** — a non-empty `blocks` array is a complete response shape on its own (no `content`/`toolCalls` required); `validateBlocks` rejects malformed arrays at load time (#274). - **Veo / Grok multi-tenant testId isolation** — native Google Veo and xAI Grok Imagine async video lifecycle mocks with record-mode live proxying and per-tenant testId isolation (#278, #282). ### Publishing Publishing is **not** performed here. The release is cut and verified; the actual publish is the manual `npm run release` step, pending maintainer action. ### Verification - `npm run build` — clean (exit 0) - `npx vitest run` — **4250/4250 passed**, 144/144 files (exit 0) - `npm run test:exports` — all 🟢 (node10 / node16 CJS+ESM / bundler, across all entry points) - Type-emitting build clean (`.d.ts` emit succeeded; no separate `tsc` script in this package)
PreviousNext