Skip to content

build(deps): bump @github/copilot and @github/copilot-sdk in /tests - #1201

Merged
JasonYeMSFT (JasonYeMSFT) merged 1 commit into
mainfrom
dependabot/npm_and_yarn/tests/multi-f54f58f9b4
Mar 9, 2026
Merged

build(deps): bump @github/copilot and @github/copilot-sdk in /tests#1201
JasonYeMSFT (JasonYeMSFT) merged 1 commit into
mainfrom
dependabot/npm_and_yarn/tests/multi-f54f58f9b4

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Mar 9, 2026

Copy link
Copy Markdown
Contributor

Bumps @github/copilot to 1.0.2 and updates ancestor dependency @github/copilot-sdk. These dependencies need to be updated together.

Updates @github/copilot from 0.0.414 to 1.0.2

Release notes

Sourced from @​github/copilot's releases.

1.0.2

2026-03-06

To commemorate GitHub Copilot CLI reaching general availability last week, we're incrementing the major version to 1.0!

  • Type 'exit' as a bare command to close the CLI
  • Ask_user form now submits with Enter key and allows custom responses in enum fields
  • Support 'command' field as cross-platform alias for bash/powershell in hook configs
  • Hook configurations now accept timeout as alias for timeoutSec
  • Fix handling of meta with control keys (including shift+enter from /terminal-setup)

0.0.423

2026-03-06

  • Users are prompted for shell commands with potentially dangerous expansion or substitution use cases, additional guardrails for malicious exploits
  • Block /share gist for EMU and GHE Cloud users with clear error messaging
  • Elicitation enum and boolean fields now require Enter to confirm a selection, with a ✓ indicator for confirmed values vs ❯ for the browsing cursor
  • MCP servers can now request users to visit a URL for out-of-band interactions such as OAuth flows or API key entry
  • Improve explore agent precision and large repository support with better context sharing
  • Diff mode displays cleanly on Windows with CRLF line endings

0.0.422

2026-03-05

  • Display request ID in authentication and authorization error messages to aid troubleshooting
  • Load personal hooks from ~/.copilot/hooks in addition to repo-level .github/hooks
  • Timeline now shows the question in a box and displays 'Making best guess on autopilot' when ask_user is auto-responded
  • Add support for GPT-5.4 model
  • Plugin cache automatically recovers from a corrupted or incomplete clone without manual intervention
  • Show a clear, actionable error message when git is not installed and a remote plugin or marketplace is used
  • Text selection persists after copying to clipboard in alt screen
  • Scroll view no longer jumps to earlier messages when scrolling during response streaming or with popups open
  • Add copy_on_select config option to auto-copy selected text to clipboard in alt screen mode
  • IME candidate windows appear at correct cursor position in CJK input
  • Add mouse scroll support to /diff in alt-screen mode
  • Reduce memory usage in alt-screen mode for long sessions
  • Diff mode now works correctly when git color.diff=always is configured
  • Opening links on Windows correctly handles URLs with & query parameters
  • @-mention file completion always reflects the current state of the working directory
  • ESC key to cancel works correctly in tmux and other non-kitty terminals
  • Click in the prompt input to reposition the text cursor
  • Add /copy command to copy the last response to clipboard
  • Links in alt-screen mode are rendered with underline styling for better visibility
  • /delegate prompts for a target remote in multi-remote repositories and clarifies confirmation text
  • GitHub MCP server stays enabled in repositories that have both Azure DevOps and GitHub remotes
  • Colons in inline code render correctly inside markdown tables
  • Pressing Ctrl+C on the help dialog now dismisses it cleanly
  • Plugin-contributed LSP servers are now loaded, started, and shown in /lsp show
  • Pressing Enter in required enum field now selects the highlighted option
  • Hide noisy todo bookkeeping queries and show dependency details in timeline

... (truncated)

Changelog

Sourced from @​github/copilot's changelog.

1.0.2 - 2026-03-06

To commemorate GitHub Copilot CLI reaching general availability last week, we're incrementing the major version to 1.0!

  • Type 'exit' as a bare command to close the CLI
  • Ask_user form now submits with Enter key and allows custom responses in enum fields
  • Support 'command' field as cross-platform alias for bash/powershell in hook configs
  • Hook configurations now accept timeout as alias for timeoutSec
  • Fix handling of meta with control keys (including shift+enter from /terminal-setup)

0.0.423 - 2026-03-06

  • Users are prompted for shell commands with potentially dangerous expansion or substitution use cases, additional guardrails for malicious exploits
  • Block /share gist for EMU and GHE Cloud users with clear error messaging
  • Elicitation enum and boolean fields now require Enter to confirm a selection, with a ✓ indicator for confirmed values vs ❯ for the browsing cursor
  • MCP servers can now request users to visit a URL for out-of-band interactions such as OAuth flows or API key entry
  • Improve explore agent precision and large repository support with better context sharing
  • Diff mode displays cleanly on Windows with CRLF line endings

0.0.422 - 2026-03-05

  • Display request ID in authentication and authorization error messages to aid troubleshooting
  • Load personal hooks from ~/.copilot/hooks in addition to repo-level .github/hooks
  • Timeline now shows the question in a box and displays 'Making best guess on autopilot' when ask_user is auto-responded
  • Add support for GPT-5.4 model
  • Plugin cache automatically recovers from a corrupted or incomplete clone without manual intervention
  • Show a clear, actionable error message when git is not installed and a remote plugin or marketplace is used
  • Text selection persists after copying to clipboard in alt screen
  • Scroll view no longer jumps to earlier messages when scrolling during response streaming or with popups open
  • Add copy_on_select config option to auto-copy selected text to clipboard in alt screen mode
  • IME candidate windows appear at correct cursor position in CJK input
  • Add mouse scroll support to /diff in alt-screen mode
  • Reduce memory usage in alt-screen mode for long sessions
  • Diff mode now works correctly when git color.diff=always is configured
  • Opening links on Windows correctly handles URLs with & query parameters
  • @-mention file completion always reflects the current state of the working directory
  • ESC key to cancel works correctly in tmux and other non-kitty terminals
  • Click in the prompt input to reposition the text cursor
  • Add /copy command to copy the last response to clipboard
  • Links in alt-screen mode are rendered with underline styling for better visibility
  • /delegate prompts for a target remote in multi-remote repositories and clarifies confirmation text
  • GitHub MCP server stays enabled in repositories that have both Azure DevOps and GitHub remotes
  • Colons in inline code render correctly inside markdown tables
  • Pressing Ctrl+C on the help dialog now dismisses it cleanly
  • Plugin-contributed LSP servers are now loaded, started, and shown in /lsp show
  • Pressing Enter in required enum field now selects the highlighted option
  • Hide noisy todo bookkeeping queries and show dependency details in timeline
  • CLI no longer hangs for minutes when working in a directory with a large number of files
  • Add --output-format json flag to emit JSONL in prompt mode for programmatic integrations
  • Add exitPlanMode.request protocol method for SDK plan approval support

... (truncated)

Commits
  • 732d8b1 Update changelog.md for version 0.0.423
  • 7aa08b3 Update changelog.md for version 0.0.422
  • 689d148 Update changelog.md for version 0.0.421
  • 119c6f8 Merge pull request #1699 from github/devm33/install-ready
  • 3b73154 Update changelog.md for version 0.0.420
  • 9596516 Update changelog.md for version 0.0.419
  • 30034c7 install: guide user to update PATH for current shell session
  • 9277ab5 Update changelog.md for version 0.0.418
  • f8396ae Merge pull request #1670 from github/RyanHecht-patch-1
  • b54e97c Update README.md
  • Additional commits viewable in compare view

Updates @github/copilot-sdk from 0.1.26 to 0.1.32

Release notes

Sourced from @​github/copilot-sdk's releases.

v0.1.32

Feature: backward compatibility with v2 CLI servers

SDK applications written against the v3 API now also work when connected to a v2 CLI server, with no code changes required.

Generated by Release Changelog Generator

Generated by Release Changelog Generator

v0.1.31

Feature: strongly-typed PermissionRequestResultKind for .NET and Go

Rather than comparing result.Kind against undiscoverable magic strings like "approved" or "denied-interactively-by-user", .NET and Go now provide typed constants. Node and Python already had typed unions for this; this brings full parity. (#631)

session.OnPermissionCompleted += (e) => {
    if (e.Result.Kind == PermissionRequestResultKind.Approved) { /* ... */ }
    if (e.Result.Kind == PermissionRequestResultKind.DeniedInteractivelyByUser) { /* ... */ }
};
// Go: PermissionKindApproved, PermissionKindDeniedByRules,
//     PermissionKindDeniedCouldNotRequestFromUser, PermissionKindDeniedInteractivelyByUser
if result.Kind == copilot.PermissionKindApproved { /* ... */ }

Other changes

  • feature: [Python] [Go] add get_last_session_id() / GetLastSessionID() for SDK-wide parity (was already available in Node and .NET) (#671)
  • improvement: [Python] add timeout parameter to generated RPC methods, allowing callers to override the default 30s timeout for long-running operations (#681)
  • bugfix: [Go] PermissionRequest fields are now properly typed (ToolName, Diff, Path, etc.) instead of a generic Extra map[string]any catch-all (#685)

Generated by Release Changelog Generator

v0.1.30

Feature: support overriding built-in tools

Applications can now override built-in tools such as grep, edit_file, or read_file. To do this, register a custom tool with the same name and set the override flag. Without the flag, the runtime will return an error if the name clashes with a built-in. (#636)

import { defineTool } from "`@github/copilot-sdk`";
const session = await client.createSession({
tools: [defineTool("grep", {
overridesBuiltInTool: true,
</tr></table>

... (truncated)

Changelog

Sourced from @​github/copilot-sdk's changelog.

v0.1.32 (2026-03-07)

Feature: backward compatibility with v2 CLI servers

SDK applications written against the v3 API now also work when connected to a v2 CLI server, with no code changes required. The SDK detects the server's protocol version and automatically adapts v2 tool.call and permission.request messages into the same user-facing handlers used by v3. (#706)

const session = await client.createSession({
  tools: [myTool],           // unchanged — works with v2 and v3 servers
  onPermissionRequest: approveAll,
});
var session = await client.CreateSessionAsync(new SessionConfig {
    Tools = [myTool],          // unchanged — works with v2 and v3 servers
    OnPermissionRequest = approveAll,
});

v0.1.31 (2026-03-07)

Feature: multi-client tool and permission broadcasts (protocol v3)

The SDK now uses protocol version 3, where the runtime broadcasts external_tool.requested and permission.requested as session events to all connected clients. This enables multi-client architectures where different clients contribute different tools, or where multiple clients observe the same permission prompts — if one client approves, all clients see the result. Your existing tool and permission handler code is unchanged. (#686)

// Two clients each register different tools; the agent can use both
const session1 = await client1.createSession({
  tools: [defineTool("search", { handler: doSearch })],
  onPermissionRequest: approveAll,
});
const session2 = await client2.resumeSession(session1.id, {
  tools: [defineTool("analyze", { handler: doAnalyze })],
  onPermissionRequest: approveAll,
});
var session1 = await client1.CreateSessionAsync(new SessionConfig {
    Tools = [AIFunctionFactory.Create(DoSearch, "search")],
    OnPermissionRequest = PermissionHandlers.ApproveAll,
});
var session2 = await client2.ResumeSessionAsync(session1.Id, new ResumeSessionConfig {
    Tools = [AIFunctionFactory.Create(DoAnalyze, "analyze")],
    OnPermissionRequest = PermissionHandlers.ApproveAll,
});

Feature: strongly-typed PermissionRequestResultKind for .NET and Go

... (truncated)

Commits

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    You can disable automated security fix PRs for this repo from the Security Alerts page.

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.


Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot added dependencies Pull requests that update a dependency file javascript Pull requests that update javascript code labels Mar 9, 2026
@JasonYeMSFT
JasonYeMSFT (JasonYeMSFT) merged commit ce3479a into main Mar 9, 2026
10 checks passed
@JasonYeMSFT
JasonYeMSFT (JasonYeMSFT) deleted the dependabot/npm_and_yarn/tests/multi-f54f58f9b4 branch March 9, 2026 16:43
Luffy Chen (XOEEst) added a commit to XOEEst/GitHub-Copilot-for-Azure that referenced this pull request Mar 11, 2026
* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule #7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule #8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (microsoft#1200)

* Collapse token analysis comment (microsoft#1147)

* update region-availability in prepare/validate/deploy skill (microsoft#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (microsoft#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.


Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (microsoft#1204)

* Add AVM (Azure Verified Modules) integration tests (microsoft#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (microsoft#1149)

* Early terminate azure-deploy, azure-validate tests (microsoft#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (microsoft#1209)

* Early terminate azure-deploy tests on deploy link (microsoft#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (microsoft#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (microsoft#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (microsoft#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (microsoft#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (microsoft#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
Luffy Chen (XOEEst) added a commit to XOEEst/GitHub-Copilot-for-Azure that referenced this pull request Mar 11, 2026
* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule #7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule #8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (microsoft#1200)

* Collapse token analysis comment (microsoft#1147)

* update region-availability in prepare/validate/deploy skill (microsoft#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (microsoft#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.

Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (microsoft#1204)

* Add AVM (Azure Verified Modules) integration tests (microsoft#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (microsoft#1149)

* Early terminate azure-deploy, azure-validate tests (microsoft#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (microsoft#1209)

* Early terminate azure-deploy tests on deploy link (microsoft#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (microsoft#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (microsoft#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (microsoft#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (microsoft#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (microsoft#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
JasonYeMSFT (JasonYeMSFT) added a commit that referenced this pull request Mar 13, 2026
…and more. (#1219)

* Fix Issue Triage workflow: add `github.token` fallback for missing `COPILOT_GITHUB_TOKEN` (#6)

* Initial plan

* Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN

The Issue Triage workflow was failing at the secret validation step because
COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback
in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication:
- agent job: validate-secret step and Execute step
- detection job: validate-secret step and Execute step

This is consistent with the existing fallback patterns in the workflow
(e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN)

Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

* Fix/observe deploy loop logic (#7)

* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule #7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule #8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (#1200)

* Collapse token analysis comment (#1147)

* update region-availability in prepare/validate/deploy skill (#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.

Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (#1204)

* Add AVM (Azure Verified Modules) integration tests (#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (#1149)

* Early terminate azure-deploy, azure-validate tests (#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (#1209)

* Early terminate azure-deploy tests on deploy link (#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* fix: expose observe prompt optimization routing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap

* chore: drop issue triage token fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: simplify deploy P0 test case guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: align microsoft-foundry version

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* test: move eval-datasets invocation tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* bump version

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
Luffy Chen (XOEEst) added a commit that referenced this pull request Mar 19, 2026
* Fix Issue Triage workflow: add `github.token` fallback for missing `COPILOT_GITHUB_TOKEN` (#6)

* Initial plan

* Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN

The Issue Triage workflow was failing at the secret validation step because
COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback
in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication:
- agent job: validate-secret step and Execute step
- detection job: validate-secret step and Execute step

This is consistent with the existing fallback patterns in the workflow
(e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN)

Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

* Fix/observe deploy loop logic (#7)

* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule #7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule #8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (#1200)

* Collapse token analysis comment (#1147)

* update region-availability in prepare/validate/deploy skill (#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.

Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (#1204)

* Add AVM (Azure Verified Modules) integration tests (#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (#1149)

* Early terminate azure-deploy, azure-validate tests (#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (#1209)

* Early terminate azure-deploy tests on deploy link (#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* fix: expose observe prompt optimization routing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap

* chore: drop issue triage token fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: simplify deploy P0 test case guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: align microsoft-foundry version

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* test: move eval-datasets invocation tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* bump version

* dataset fix (#9)

* dataset fix

* fix: restore foundry dataset guidance

Restore the explicit seed dataset registration guidance in the deploy skill and align dataset docs with the current evaluation_dataset_create MCP surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: add codeowner for microsoft-foundry

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address foundry PR review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: normalize foundry dataset versioning docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Deduplicate Foundry dataset registration docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* use sdk for eval results

* Fix comments

* Align observe test with SDK guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant Foundry MCP tips

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix dataset manifest version examples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Align eval-datasets metadata docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Merge remote-tracking branch 'upstream/main'

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
Barbara 4bes (Ba4bes) pushed a commit to Ba4bes/GitHub-Copilot-for-Azure that referenced this pull request Apr 24, 2026
…icrosoft#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.


Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Barbara 4bes (Ba4bes) pushed a commit to Ba4bes/GitHub-Copilot-for-Azure that referenced this pull request Apr 24, 2026
…and more. (microsoft#1219)

* Fix Issue Triage workflow: add `github.token` fallback for missing `COPILOT_GITHUB_TOKEN` (microsoft#6)

* Initial plan

* Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN

The Issue Triage workflow was failing at the secret validation step because
COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback
in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication:
- agent job: validate-secret step and Execute step
- detection job: validate-secret step and Execute step

This is consistent with the existing fallback patterns in the workflow
(e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN)

Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

* Fix/observe deploy loop logic (microsoft#7)

* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule microsoft#7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule microsoft#8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (microsoft#1200)

* Collapse token analysis comment (microsoft#1147)

* update region-availability in prepare/validate/deploy skill (microsoft#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (microsoft#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.

Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (microsoft#1204)

* Add AVM (Azure Verified Modules) integration tests (microsoft#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (microsoft#1149)

* Early terminate azure-deploy, azure-validate tests (microsoft#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (microsoft#1209)

* Early terminate azure-deploy tests on deploy link (microsoft#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (microsoft#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (microsoft#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (microsoft#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (microsoft#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (microsoft#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* fix: expose observe prompt optimization routing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap

* chore: drop issue triage token fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: simplify deploy P0 test case guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: align microsoft-foundry version

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* test: move eval-datasets invocation tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* bump version

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
Barbara 4bes (Ba4bes) pushed a commit to Ba4bes/GitHub-Copilot-for-Azure that referenced this pull request Apr 24, 2026
* Fix Issue Triage workflow: add `github.token` fallback for missing `COPILOT_GITHUB_TOKEN` (microsoft#6)

* Initial plan

* Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN

The Issue Triage workflow was failing at the secret validation step because
COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback
in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication:
- agent job: validate-secret step and Execute step
- detection job: validate-secret step and Execute step

This is consistent with the existing fallback patterns in the workflow
(e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN)

Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

* Fix/observe deploy loop logic (microsoft#7)

* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule microsoft#7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule microsoft#8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (microsoft#1200)

* Collapse token analysis comment (microsoft#1147)

* update region-availability in prepare/validate/deploy skill (microsoft#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (microsoft#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.

Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (microsoft#1204)

* Add AVM (Azure Verified Modules) integration tests (microsoft#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (microsoft#1149)

* Early terminate azure-deploy, azure-validate tests (microsoft#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (microsoft#1209)

* Early terminate azure-deploy tests on deploy link (microsoft#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (microsoft#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (microsoft#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (microsoft#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (microsoft#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (microsoft#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* fix: expose observe prompt optimization routing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap

* chore: drop issue triage token fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: simplify deploy P0 test case guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: align microsoft-foundry version

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* test: move eval-datasets invocation tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* bump version

* dataset fix (microsoft#9)

* dataset fix

* fix: restore foundry dataset guidance

Restore the explicit seed dataset registration guidance in the deploy skill and align dataset docs with the current evaluation_dataset_create MCP surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: add codeowner for microsoft-foundry

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address foundry PR review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: normalize foundry dataset versioning docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Deduplicate Foundry dataset registration docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* use sdk for eval results

* Fix comments

* Align observe test with SDK guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant Foundry MCP tips

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix dataset manifest version examples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Align eval-datasets metadata docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Merge remote-tracking branch 'upstream/main'

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
Luffy Chen (XOEEst) added a commit that referenced this pull request Apr 30, 2026
* Fix Issue Triage workflow: add `github.token` fallback for missing `COPILOT_GITHUB_TOKEN` (#6)

* Initial plan

* Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN

The Issue Triage workflow was failing at the secret validation step because
COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback
in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication:
- agent job: validate-secret step and Execute step
- detection job: validate-secret step and Execute step

This is consistent with the existing fallback patterns in the workflow
(e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN)

Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

* Fix/observe deploy loop logic (#7)

* Fix broken auto-create evaluators step in deploy/observe loop

The 'Auto-create evaluators & evaluation dataset' step was being skipped
when the monolithic agent-observability-loop skill was split into separate
deploy and observe skills. Neither skill owned the auto-create step,
causing post-deploy users to jump directly to evaluation.

Changes:
- deploy.md: Replace generic 'set up evaluation?' prompt with automatic
  6-step evaluator & dataset creation matching the reference behavior
- observe.md: Add Loop Overview, fix entry points to route post-deploy
  users through auto-setup, add evaluator existence check
- deploy-and-setup.md: Make auto-create primary content, demote deploy
  section to prerequisites

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add content tests for observe/deploy loop logic

Tests verify:
- observe.md has Loop Overview, post-deploy entry points, evaluator
  existence checks, behavioral rules, and all reference files
- deploy.md has auto-create evaluators section that is automatic (not
  optional), includes evaluator categories, LLM-judge, artifact
  persistence, and routes to observe skill Step 2
- deploy-and-setup.md has auto-create as primary content with proper
  evaluator selection, dataset generation, and user prompt

49 tests total (29 observe + 20 deploy), all passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: trigger CI checks

* Fix

* add local dataset gen enforcement

* Merge

* feat: prefer monitor_resource_log_query and local datasets

- Replace azure-kusto delegation with monitor_resource_log_query for
  App Insights KQL queries in trace.md and troubleshoot.md
- Mark evaluation_dataset_create as not available (MCP upload not ready)
- Replace server-side dataset sections with local JSONL workflow
- Update mcp-gap-analysis.md to reflect practical tool availability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make dataset upload restriction more agent-proof

- Add Do NOT section at top of trace-to-dataset.md (before Overview)
- Add behavioral rule #7 to eval-datasets.md: never upload to cloud
- Remove Option A/B structure; Step 4 is now local JSONL only
- Eliminates subtle strikethrough formatting that agents miss

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix link

* fix: make auto-create evaluators an explicit numbered step

- Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate
- Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate
- Both link to existing After Deployment section as implementation
- Prevents agents from treating evaluator setup as optional appendix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: add dataset update loop with optimization guardrails

- Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval)
  to dataset-versioning.md after Creating a New Version
- Add guardrails: never remove dataset rows or weaken evaluators
  to recover scores after dataset expansion
- Add same guardrail to observe optimize-deploy.md Step 6
- Add behavioral rule #8 to eval-datasets.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add subscription parameter warning to trace-related skills

Always pass subscription explicitly to Azure MCP tools like
monitor_resource_log_query — they don't extract it from resource IDs.
Added to trace.md, troubleshoot.md, and trace-to-dataset.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make customEvents-to-traces eval correlation more obvious

- Add Key Concept section to trace-to-dataset.md explaining that
  eval results live in customEvents (not dependencies) and the
  join key is gen_ai.response.id
- Add table showing dependencies vs customEvents join pattern
- Cross-reference trace skill's eval-correlation.md from both
  trace-to-dataset.md and eval-datasets.md Related Skills

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve cross-references and add KQL parse_json warning

1. Add parse_json(customDimensions) warning to Do NOT section
2. Add Related References section with skill-root paths
3. Add skill-root path hints to all cross-skill links
4. Add observe + trace to SKILL.md sub-skill routing table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve hosted agent KQL patterns and content extraction

- Add Hosted Agent Harvest template (requests→dependencies join)
- Fix Hosted Agent Attributes: appear on both requests and traces
- Add gen_ai.agent.name duality callout (Foundry name vs class name)
- Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries
- Document gen_ai.input.messages/gen_ai.output.messages as content source
- Add operation_ParentId join example to Span Correlation section
- Update search-traces.md hosted agent query to use requests entry point

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: improve trace sub-skills for hosted agent KQL patterns

- search-traces: fix hosted agent query to group by operation_ParentId
- conversation-detail: add content extraction from invoke_agent spans
  (gen_ai.input.messages / gen_ai.output.messages)
- analyze-failures: add hosted agent gen_ai.agent.name duality warning
  and hosted agent variant query using requests→dependencies join
- analyze-latency: same hosted agent warning and variant query
- kql-templates: expand requests table description as preferred entry
  point; add gen_ai.input/output.messages to attributes table
- trace.md: reword rule 6 to clarify hosted vs prompt agent filtering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore routing keywords and update trigger snapshots

- Add back critical routing keywords to SKILL.md description (578→779 chars):
  role assignment, permissions, capacity, region, deployment failure,
  AI Services, Cognitive Services, provision, knowledge index,
  monitoring, customize, onboard, availability
- Update trigger test snapshots for new keyword set (24 snapshots)
- Fix deploy trigger test: Docker IS our capability (remove false negative)
- Fix customize-deployment tests: ensure prompts have ≥2 keyword matches
- Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add 'create AI Services' to description for resource/create test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump microsoft-foundry version to 1.0.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(eval-datasets): enable Foundry dataset sync via MCP tools

- Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using
  evaluation_dataset_create with connectionName and project_connection tools
- Add server-side version discovery via evaluation_dataset_versions_get
- Add dual experiment types to dataset-comparison (agent vs dataset comparison)
- Update mcp-gap-analysis: mark resolved tools, update workarounds
- Add AzureBlob to project connections reference
- Bump microsoft-foundry version to 1.0.3
- Fix upstream section heading changes in unit tests
- Update trigger snapshots for upstream keyword changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(dataset-comparison): focus on dataset-version comparison only

Remove agent comparison experiment type from dataset-comparison flow.
Agent comparison belongs in the observe/eval loop, not the dataset skill.
Update all examples to use dataset versions as baseline/treatment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove Playwright MCP server until skills require it (#1200)

* Collapse token analysis comment (#1147)

* update region-availability in prepare/validate/deploy skill (#1083)

* update region-availability in prepare/deploy skill

* update

* update

* fix

* update date

* Update plugin/skills/azure-deploy/references/region-availability.md

* fix ci failure

* bump version

* build(deps): bump @github/copilot and @github/copilot-sdk in /tests (#1201)

Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together.

Updates `@github/copilot` from 0.0.414 to 1.0.2
- [Release notes](https://github.com/github/copilot-cli/releases)
- [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md)
- [Commits](github/copilot-cli@v0.0.414...v1.0.2)

Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32
- [Release notes](https://github.com/github/copilot-sdk/releases)
- [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32)

---
updated-dependencies:
- dependency-name: "@github/copilot"
  dependency-version: 1.0.2
  dependency-type: indirect
- dependency-name: "@github/copilot-sdk"
  dependency-version: 0.1.32
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* specify application path in prompt (#1204)

* Add AVM (Azure Verified Modules) integration tests (#1171)

* Add AVM (Azure Verified Modules) integration tests

Add 3 integration tests validating the AVM module selection hierarchy
for Bicep infrastructure generation:

- avm-module-priority: Verifies AVM modules prioritized over non-AVM
- avm-fallback-behavior: Verifies fallback stays within AVM ecosystem
- avm-azd-pattern-preference: Verifies AZD pattern modules preferred

Tests validate that the azure-deploy skill enforces the mandatory
AVM selection order: Pattern modules > Resource modules > Utility
modules, and never falls back to non-AVM alternatives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add output assertions to AVM integration tests

Address Copilot review feedback: add keyword-based output
assertions using getAllAssistantMessages/getAllToolText to
verify agent responses contain AVM hierarchy terms, not just
skill invocation. Includes non-AVM fallback negative check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Strengthen AVM test output assertions per Copilot review

- Split keyword checks into critical-term + context assertions
- Add resource-before-utility ordering assertion for fallback test
- Expand non-AVM negative check to use regex patterns
- Require core keywords (avm+pattern, azd+pattern) explicitly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check

- Add hierarchy ordering assertion to test 1 (pattern before resource/utility)
- Make non-AVM detection context-aware: skip matches preceded by negation words
  (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive)
- Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor: move AVM integration tests to avm/ subdirectory

Move tests/azure-deploy/avm-integration.test.ts to
tests/azure-deploy/avm/integration.test.ts so the file matches the
**/integration.test.ts glob used by the custom ESLint rule
(integration-test-name) and follows the subdirectory convention
established by tests/microsoft-foundry/ (e.g. foundry-agent/).

Import paths updated from ../utils/ to ../../utils/ to reflect the
new depth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address round 4 Copilot review feedback

- Add 'fall back'/'fall-back' keyword variants for resilience
- Extend non-AVM negation check to also scan following context
- Use regex for AZD ordering assertion to match plural/prefixed variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update github workflows to use best practices (#1149)

* Early terminate azure-deploy, azure-validate tests (#1205)

Add comment for early termination to help AI grader

* Replace script inline parameters with env var (#1209)

* Early terminate azure-deploy tests on deploy link (#1208)

* Early terminate azure-deploy tests on deploy link

* Fix lint issue

* Reduce char count of existing skills (#1210)

* Reduce char count of existing skills

* Update ci tests and snapshots

* Enhance benchmark ci run script (#1176)

* Add msbench_benchmarks repo clone to get model definition

* Remove unused vars

* Use mcp-pr repo before MI has access to msbench-benchmarks repo

* Address copilot feedback

* Change back to msbench-benchmarks repo

* Get ADO token for repo clone

* Fix line continuation character

* Add run for all interested models

* Extract run IDs

* Fix yaml format issue

* Schedule it to run nightly

* Address copilot feedbacks

* formalize .foundry and multi-environment support

* fix

* Feature/azure quotas (#1137)

* update for using azure-quotas in skill

* test update

* unit test update

* path update

* add skill in skills.json

* skills.json update

* reduce the text

* version update

* skill version

* skill description update

* reduce text size

* 1.0.4 for next prepare version

* upload snap shot

* update version

* test update

---------

Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (#1213)

Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3.
- [Release notes](https://github.com/steveukx/git-js/releases)
- [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
- [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git)

---
updated-dependencies:
- dependency-name: simple-git
  dependency-version: 3.32.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve azure-compliance invocation rate (#1214)

* Improve azure-compliance invocation rate

* Race condition free report writing

* Fix debug logging for report location

* Bump skill version

* Fix suffix base value

* fix

* llm judge model and eval group improvement

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

* fix: expose observe prompt optimization routing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap
#	tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap

* chore: drop issue triage token fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: simplify deploy P0 test case guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: align microsoft-foundry version

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* test: move eval-datasets invocation tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments

* bump version

* dataset fix (#9)

* dataset fix

* fix: restore foundry dataset guidance

Restore the explicit seed dataset registration guidance in the deploy skill and align dataset docs with the current evaluation_dataset_create MCP surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: add codeowner for microsoft-foundry

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address foundry PR review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: normalize foundry dataset versioning docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Deduplicate Foundry dataset registration docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* use sdk for eval results

* Fix comments

* Align observe test with SDK guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant Foundry MCP tips

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix dataset manifest version examples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Align eval-datasets metadata docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Merge remote-tracking branch 'upstream/main'

* feat(microsoft-foundry): refresh agent evaluation workflows

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Enforce LF line endings for Markdown files

Windows clones with core.autocrlf=true convert SKILL.md to CRLF in the
working tree, breaking unit tests like airunway-aks-setup's
'frontmatter has no tabs' check whose regex (/^-{3}\n.../) does not
match across \r\n.

Forcing eol=lf for *.md keeps line endings consistent across platforms
and prevents test failures on Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): rename "test suite" to "evaluation suite" and add regression purpose

Addresses PR #2092 review comments from @jongio:

1. Standardize terminology on "evaluation suite" to match the
   evaluationSuites[] schema field. Renames human-language "test suite"
   and "test-suite" mentions across deploy.md, observe.md, analyze-results.md,
   evaluate-step.md, compare-iterate.md, cicd-monitoring.md, and the
   agent-metadata-contract.md guidance section/heading.

2. Add `regression` to the `purpose` values table in
   agent-metadata-contract.md so the suggested values match the
   `trace-regression-suite` example (`purpose: regression`) and the
   analyze-results.md prioritization rule (`purpose=regression`).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): document metadata-file fallback when default is absent

Addresses PR #2092 Copilot review (#pullrequestreview-4195041793,
discussion r3159385842) by extending Workflow Rule #3 in
agent-metadata-contract.md to describe what happens when
.foundry/agent-metadata.yaml is missing:

- Use the only matching .foundry/agent-metadata.<env>.yaml sidecar
  when exactly one exists.
- Require the user to choose the metadata file when multiple sidecar
  files exist and no explicit file/path was provided.

Keeps the contract symmetric with Workflow Rule #2 (multiple agent
roots also require a user choice) and removes the previous ambiguity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Chris Harris <charris@microsoft.com>
Co-authored-by: JasonYeMSFT <chuye@microsoft.com>
Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>
Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com>
Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file javascript Pull requests that update javascript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant