Merge main into branch - #2
Merged
Merged
Conversation
* chore: add new plugin bootstrap script * fix lint * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * write blank mcp config * use pluginName constant variable add mcpServers to base plugin manifest * skip hooks bootstrap --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
) * feat: add azure-validate workflow.ps1 step-driver script Replace the inline nine-step table in SKILL.md with a workflow.ps1 script that walks the agent through each validation step one at a time, tracking progress via completedStep in .azure/validate-status.json. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727 * feat: drive azure-validate workflow via -CompletedStep param workflow.ps1 now accepts a -CompletedStep parameter and writes the progress to .azure/validate-status.json itself, instead of instructing the agent to edit the file. The agent starts the workflow by calling the script with no -CompletedStep, and each response tells it which value to pass next. This eliminates a per-step tool call by the agent. SKILL.md updated to document the new invocation pattern. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727 * feat: add bash workflow.sh for azure-validate Add a Mac/Linux bash equivalent of workflow.ps1 that drives the azure-validate workflow step-by-step via --completed-step, mirroring the PowerShell script's output exactly. Link both scripts (and restore recipe/role reference links) from SKILL.md so orphan detection passes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727 * fix: address workflow script review feedback - Validate the workspace path exists and is a directory, failing early with a clear message instead of a later New-Item/Set-Content error. - Emit intended Markdown backticks in agent-facing output by using single-quoted PowerShell strings (backticks were being consumed as PowerShell escapes); mirror the same output in workflow.sh. - Write validate-status.json as UTF-8 without BOM so non-PowerShell JSON consumers can parse it (Windows PowerShell 5.1 defaulted to UTF-16LE). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727 --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727
* doc: plugin structure doc * update new plugin instruction
* chore: fix outdated mentions of old plugin folder structure * fix skill name in token/tool usage report * fix mentions of output/skills
…script (microsoft#2936) * feat: replace AKS baseline diagnostic sweep with aks-baseline script Adds cross-platform aks-baseline.sh/.ps1 that runs the read-only AKS cluster baseline sweep once and prints a single labeled digest (provisioning state, node pools, activity log, node readiness, unhealthy pods, kube-system health, warning events). Replaces the duplicated inline command blocks in general-diagnostics.md, aks-troubleshooting.md, and command-flows.md with script references. Fixes microsoft#2506 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * fix: address PR review feedback on aks-baseline scripts - sh: usage() accepts an exit code (--help exits 0); validate a value exists before consuming each option so a missing value no longer loops - sh: set pipefail inside the warning-events subshell so kubectl failures are surfaced by run() - ps1: drop mandatory params (avoids interactive prompts in non-interactive runs); explicitly validate ResourceGroup/Cluster and exit early with a clear message - ps1: remove redundant \Continue assignment (default is Continue) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * test: add eval that exercises the aks-baseline script Adds a response-quality stimulus to the azure-diagnostics eval suite that supplies concrete cluster identity and asserts (via the tool-calls grader) that the agent actually runs aks-baseline.sh/.ps1 rather than pasting the individual az/kubectl commands. No real cluster is required — each script step is guarded, so the sweep completes and emits its digest regardless. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * test: early-terminate aks-baseline eval when the script tool call starts Switches the script-execution stimulus to earlyTerminate on tool-call-match so the run aborts the moment the aks-baseline shell tool call starts, before the script actually executes. The triggering start event is still recorded, so the tool-calls grader confirms the agent chose to run the script; drops the completed grader per the early-terminate convention. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * fix: make aks-baseline eval gradable under early termination The eval early-terminates the moment the aks-baseline script tool call starts, so no tool_result is ever emitted. The built-in tool-calls grader only counts a required tool on its tool_result, so it could never match an early-terminated call. Its inline (?i) regex flag also crashed the custom executor's tag-helpers, which compile earlyTerminate patterns with plain new RegExp(). - Drop (?i) / anchors from the earlyTerminate patterns (JS RegExp has no inline flags), matching the azure-deploy convention. - Add a small custom tool-call-started grader that matches the tool-call start event, and register it in vally-graders.ts. - Switch the stimulus to the new grader and make the prompt directive so the agent routes straight to the script (no real cluster exists in the test subscription). Verified locally: all 3 graders pass (100%); the run terminates at the script tool-call start with no tool_result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * refactor: grade aks-baseline eval with built-in tool-calls grader Replace the custom tool-call-started grader with the built-in tool-calls grader, matching the pattern used by the azure-validate e2e suite. The built-in grader only matches a tool call once its result is recorded, so switch earlyTerminate from tool-call-match (aborts at the call's start, before completion) to tool-call-result (fires once the call completes). The aks-baseline script is read-only and fully guarded, so against a missing cluster every step fails fast and the sweep still exits cleanly in seconds. The call completes -> its tool_result is recorded -> tool-calls equired matches; termination then stops the run before any follow-up turns. No custom grader needed. Verified locally: all 3 graders pass (100%); the script tool call has a matching tool_result and the run ends at 2 turns. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * fix: resolve upstream merge — relocate aks-baseline scripts and migrate eval to requiredSkills The multi-plugin restructure (microsoft#2872) moved plugin content to plugins/azure-skills/ and changed the eval executor to load skills only for stimuli declaring a per-stimulus requiredSkills tag. This merge: - Moves the new aks-baseline.sh/.ps1 scripts, orphaned by the merge at the old plugin/ path, to plugins/azure-skills/skills/azure-diagnostics/scripts/. - Adds requiredSkills: [azure-diagnostics] to all azure-diagnostics eval stimuli so the skill is loaded into the agent (without it, the agent gets no skills and every run reports 'Skills used 0'). Verified: build, frontmatter, references, validate-stimulus, typecheck, lint all pass; the aks-baseline script eval passes 3/3 (100%). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 * fix: broaden aks-baseline unhealthy-pod check to catch container-level failures Address PR review: the phase-based field selector (status.phase!=Running,!=Succeeded) missed pods that stay in phase Running while a container fails — CrashLoopBackOff, ImagePullBackOff, readiness-probe failures, and high restart counts. Step 5 now lists all pods (kubectl get pods -A -o wide) and flags a pod when it is not Ready, has a non-Running/non-terminal STATUS, or has a high restart count. Terminal pods (Completed/Succeeded) are excluded so finished jobs are not falsely reported. Both the bash and PowerShell scripts use the same READY/STATUS/RESTARTS logic and were verified to flag identical rows. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72 --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
…crosoft#2833) * feat: add azure-deploy e2e eval fixtures and suite Identified 20 integration tests in tests/azure-deploy/integration.test.ts that started from empty workspaces and asked the agent to both create and deploy applications. This conflates azure-prepare (scaffolding) and azure-deploy (running azd up) responsibilities. Add per-scenario fixtures under evals/azure-deploy/fixture/ so each vally stimulus provides a pre-built azure.yaml + infra/ and the agent is only evaluated on deployment execution: Fixtures added: - swa-bicep/ Static Web App with Bicep infra - app-service-bicep/ App Service with Bicep infra - functions-bicep/ Azure Functions with Bicep infra - dts-bicep/ Durable Task Scheduler with Bicep infra (DTS-specific Microsoft.DurableTask/* resources + role 0ad04412) - container-apps-bicep/ Container Apps with Bicep infra (two-phase pattern: SystemAssigned identity, placeholder image, AcrPull 7f951dda) - swa-terraform/ Static Web App with Terraform infra - app-service-terraform/ App Service with Terraform infra - functions-terraform/ Azure Functions with Terraform infra - container-apps-terraform/ Container Apps with Terraform infra (SystemAssigned identity, AcrPull, lifecycle ignore_changes) New eval suite: evals/azure-deploy/e2e-eval.yaml Migrates all 20 vanilla/terraform integration tests. Each stimulus: - Loads the appropriate fixture into the agent environment - Prompts "deploy this prepared app" (not "create and deploy") - Uses earlyTerminate on azd up/provision - Graders check azure-deploy skill invocation + azd command execution - Container Apps stimuli also verify correct IaC patterns in fixture files Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * feat: add azure-deploy routing eval suite Migrate describe('skill-invocation') tests from tests/azure-deploy/integration.test.ts to routing-eval.yaml. Four routing stimuli, each with a matching pre-built fixture so the agent has concrete project context: - 'Deployment Prompt' (swa-bicep fixture, tier: smoke) - 'Publish to Azure Prompt' (swa-bicep fixture, tier: full) - 'Azure Functions Deployment Prompt' (functions-bicep fixture, tier: full) - 'Post-Deployment Role Assignment Prompt' (container-apps-bicep, tier: full) All use earlyTerminate on skill-call or tool-call-count:10, and skill-invocation grader requiring azure-deploy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * feat: migrate remaining azure-deploy tests to vally evals output-eval.yaml: - Add 'AZD Pattern Module Priority - Container App' stimulus from tests/azure-deploy/avm/integration.test.ts describe('avm-azd-pattern-preference'). Graders check AZD pattern keyword presence and pattern-before-resource ordering. deploy-eval.yaml: - Bump default timeout 45m → 55m to cover brownfield scenarios. - Add 13 brownfield stimuli across describe('brownfield-dotnet'), describe('brownfield-javascript'), describe('brownfield-python'): brownfield-dotnet (8): - Brownfield .NET - eShop (git clone dotnet/eShop) - Brownfield .NET - MvcMovie 10 (sparse checkout AspNetCore.Docs) - Brownfield .NET Aspire - Azure Functions - Brownfield .NET Aspire - Client Apps Integration - Brownfield .NET Aspire - Container Build (+file-matches on two-phase Bicep pattern) - Brownfield .NET Aspire - Custom Resources (No Deploy) (disallows azd up) - Brownfield .NET Aspire - Database Containers - Brownfield .NET Aspire - Orleans Voting (+output-matches for container env vars) brownfield-javascript (3): - Brownfield JavaScript - nodejs-demoapp (git clone benc-uk/nodejs-demoapp) - Brownfield .NET Aspire - JavaScript - Brownfield .NET Aspire - Node brownfield-python (2): - Brownfield Python - Flask Calculator (git clone UltiRequiem/flask-calculator) - Brownfield .NET Aspire - Python All brownfield stimuli use environment.commands (git clone / sparse-checkout), earlyTerminate on azd up/provision, and require azure-prepare + azure-deploy skill invocations — matching the full pipeline tested by the original Jest tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * Reorganize eval suites * eval: add azd deployment tool hint to all azure-deploy eval prompts Add 'Use azd as the deployment tool.' to all 40 stimuli across: - deploy-eval.yaml (33 prompts) - routing-eval.yaml (4 prompts) - output-eval.yaml (3 prompts) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * limit to 1 output test per run * eval: add category tags to azure-deploy deploy-eval stimuli Map each stimulus back to its original Jest describe group: - vanilla-static-web-apps-deploy (2) - vanilla-app-service-deploy (2) - vanilla-azure-functions-deploy (3) - durable-task-scheduler-deploy (1) - vanilla-azure-container-apps-deploy (2) - terraform-static-web-apps-deploy (2) - terraform-app-service-deploy (2) - terraform-azure-functions-deploy (3) - terraform-azure-container-apps-deploy (3) - brownfield-dotnet (8) - brownfield-javascript (3) - brownfield-python (2) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * eval: switch earlyTerminate to tool-call-result in azure-deploy deploy-eval 32 stimuli had both a tool-calls grader checking for azd up/provision and an earlyTerminate using tool-call-match on the same command. tool-call-match fires when the call starts (before execution), so the grader may not see a completed call in the trajectory. Switching to tool-call-result ensures early termination happens after the tool call completes, allowing the tool-calls grader to evaluate the result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * eval: change earlyTerminate to detect deploy URL in assistant output Replace azd up/provision tool-call-result earlyTerminate with assistant-message-match on Azure deployment link patterns: - *.azurewebsites.net (App Service) - *.azurestaticapps.net (Static Web Apps) - *.azurecontainerapps.io (Container Apps) - *.web.core.windows.net (Storage static website) This matches the original shouldEarlyTerminateForCompletedDeployment logic in tests/azure-deploy/utils.ts. Tests now run through actual deployment and terminate when the agent surfaces a live app URL. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * eval: add takeScreenshot tag to azure-deploy deploy-eval stimuli Add has-deployment-url takeScreenshot condition to the 17 stimuli that had takeScreenshot configured in the original Jest tests. Screenshot triggers when the agent outputs any Azure deployment link: - *.azurewebsites.net (App Service) - *.azurestaticapps.net (Static Web Apps) - *.azurecontainerapps.io (Container Apps) - *.web.core.windows.net (Storage static website) Stimuli with takeScreenshot (17): - SWA Bicep (2), App Service Bicep (2) - SWA Terraform (2), App Service Terraform (2) - Brownfield .NET: eShop, MvcMovie 10, Aspire Azure Functions, Aspire Client Apps Integration - Brownfield JavaScript: nodejs-demoapp, Aspire JS, Aspire Node - Brownfield Python: Flask Calculator, Aspire Python Stimuli without takeScreenshot (16): - Azure Functions (Bicep/Terraform), DTS, Container Apps (Bicep/Terraform), Brownfield .NET: Container Build, Custom Resources, DB Containers, Orleans Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * update CI test command * eval: add application code to DTS fixture for order processing workflow The dts-bicep fixture was missing Node.js app code. The azure.yaml declares language:js host:function, so azd expects deployable source. Added (Azure Functions v4 model): - package.json — @azure/functions v4, @azure/durabletask deps - src/functions/orderOrchestrator.js — orchestrator coordinating 3 sequential steps: validateOrder → processPayment → sendConfirmation - src/functions/orderActivities.js — the three activity functions - src/functions/startOrder.js — HTTP POST trigger to start the pipeline Updated deploy-eval.yaml to include all new files in the stimulus environment.files list. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * eval: rename whiteboard to test-sample-app in SWA fixture HTML Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4 * restore test prompts * increase all max_turns to 80 * relax IaC check path * remove incorrect bicep check fix acr endpoint check location * add deployed link grader * clear eval suites * disallow provision command for no deploy test * properly dispatch azure-deploy test runs --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01827688-80a4-46ef-90c2-15e7de402bc4
…tes (microsoft#3005) * feat: migrate azure-cloud-migrate integration tests to vally eval suites Adds evals/azure-skills/azure-cloud-migrate/eval.yaml with 8 stimuli migrated from tests/azure-cloud-migrate/integration.test.ts: Brownfield (tier: full, runs: 1): - Face-blur Lambda migration — clones aws-samples/serverless-face-blur-service, runs migration prompt + follow-up, checks skill invocation and output files (migration-status.md, migration-assessment-report.md, src/{app,detectFaces,blurFaces}.js) - Webapp Lambda migration — clones aws-samples/lambda-refarch-webapp, checks skill invocation and migration-status.md / migration-assessment-report.md Routing (tier: smoke/full, runs: 5, earlyTerminate on skill-call): - Spring Boot to ACA compatibility assessment (smoke) - Spring Boot JAR containerization to ACA (full) - AWS Fargate ECS tasks to ACA (smoke) - ECS Fargate containers to ACA (full) - GKE Kubernetes workloads to ACA (smoke) - Kubernetes manifest conversion to ACA config (full) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * delete jest code use vally test in CI * remove unnecessary comments * fix comment --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* chore: use msft approved npm feed * remove min-release-age and dependabot cooldown
…t#3018) * eval: migrate foundry skill e2e workflow to azure pipelines * chore: clean
…ug logging (microsoft#3021) * fix: resolve Claude hooks.json validation error Claude additively loads the default hooks/hooks.json regardless of the manifest's hooks path, so the Copilot-format file there failed 'claude plugin validate'. Rename it to hooks/copilot-hooks.json and reference it explicitly from .plugin/plugin.json (Copilot/VS Code). Nothing now sits at the shared default path, so each client uses its own hooks file. Addresses the hooks portion of microsoft#2957; the unpinned npx launcher is handled separately. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7ace74d0-93aa-4f8c-a9ef-0a928b7d50ad * feat: add telemetry debug logging and local plugin-root skill detection Add opt-in debugging to the telemetry hook scripts, controlled by the AZURE_SKILLS_TELEMETRY_LOG_DIR env var: raw JSON inputs are written to a raw-input/ subdirectory and MCP args are appended to telemetry.log. Also honor AZURE_SKILLS_PLUGIN_ROOT so skills loaded via --plugin-dir are recognized for reference_file_read events, and generalize reference-path extraction to match any skills/ root. Applied symmetrically to track-telemetry.ps1 and track-telemetry.sh. Related to microsoft#2957 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7ace74d0-93aa-4f8c-a9ef-0a928b7d50ad * fix: address PR review feedback on telemetry debug logging - Use printf instead of echo for raw-input dumps in track-telemetry.sh so JSON is written losslessly without escape-sequence interpretation. - Use local time (no trailing Z) for the debug log timestamp in both scripts. - Fix a stale comment that referenced AZURE_SKILLS_TELEMETRY_LOG instead of AZURE_SKILLS_TELEMETRY_LOG_DIR. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7ace74d0-93aa-4f8c-a9ef-0a928b7d50ad * docs: update hooks manifest location to copilot-hooks.json Reflect the rename of the Copilot/VS Code hooks manifest away from the default hooks/hooks.json path and explain why (Claude's additive hooks discovery), per issue microsoft#2957. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7ace74d0-93aa-4f8c-a9ef-0a928b7d50ad * fix: bootstrap Copilot plugin manifest with explicit hooks property New plugins scaffolded by bootstrap.ts now set \hooks: ./hooks/copilot-hooks.json\ so no Copilot-format manifest sits at the default hooks/hooks.json path, consistent with issue microsoft#2957. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7ace74d0-93aa-4f8c-a9ef-0a928b7d50ad --------- Copilot-Session: 7ace74d0-93aa-4f8c-a9ef-0a928b7d50ad
…icrosoft#2935) * feat: extract azure-diagnostics dump-everything blocks into scripts Replace the inline chained 'dump everything' diagnostic blocks in the container-apps and app-service references with maintained, cross-platform scripts (.sh + .ps1) under the skill's scripts/ folder. Scripts collect and label diagnostic sections (config, revisions/deployments, recent logs, etc.) and print a summary line; interpretation stays in agent prose. READMEs now link to the scripts with sample invocations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd * fix: drop redundant -o json from containerapp diagnostics script The Azure CLI already defaults to JSON output, and the original inline block did not specify -o json. Removing it keeps the script's behavior identical to the original. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd * fix: harden diagnostics script argument parsing Bash: validate that a value follows each flag (clear error instead of set -u 'unbound variable') and reject unknown options instead of silently treating typos as positional args. PowerShell: drop [Parameter(Mandatory)] (which prompts interactively when omitted) in favor of an explicit check that errors and exits early in non-interactive scenarios. Addresses review comments on PR microsoft#2935. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd * test: add eval verifying agent invokes diagnostics scripts Adds a vally response-quality eval (evals/azure-diagnostics/script-invocation.eval.yaml) with two stimuli that check the agent both routes to azure-diagnostics and asks to run the bundled containerapp-diagnostics / appservice-diagnostics script (via a tool-calls grader on the shell command). Mirrors the existing azure-quotas check-quota pattern. Does not require live Azure resources — it grades the script invocation, not its output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd * test: early-terminate diagnostics eval at script invocation The agent invokes the bundled diagnostic script but the run is aborted at that tool call, so the script is never actually executed. Drops the completed grader (forbidden alongside earlyTerminate). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd * test: set default runs to 1 for diagnostics script-invocation eval Per-stimulus runs override is a no-op today (microsoft/vally#430); move runs:1 to defaults and drop the ineffective per-stimulus overrides. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd * test: terminate diagnostics eval on tool-call-result so grader matches The built-in tool-calls grader only counts a required match when it sees the tool_result (execution_complete). Terminating at tool-call-match (execution_start) left a dangling tool_call the grader could not match, so switch both stimuli to tool-call-result. The bundled scripts are read-only az queries that no-op without resources, so running them once is harmless and the run still aborts immediately after. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd --------- Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
…cript (microsoft#2932) * feat: replace azure-diagnostics messaging connectivity probe with a script Adds cross-platform test-messaging-connectivity.{sh,ps1} scripts that probe DNS resolution, HTTPS reachability, and messaging TCP ports (AMQP 5671/5672, HTTPS 443, optional Kafka 9093) for a Service Bus / Event Hubs namespace, emitting one normalized report. Replaces the inline curl/nslookup probe in service-troubleshooting.md with markdown-linked references and examples. Fixes microsoft#2511 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * test: add eval covering messaging connectivity probe script Adds an output-content stimulus to evals/azure-diagnostics/eval.yaml that presents a 'cannot connect at all' Service Bus namespace scenario and asserts the skill surfaces the test-messaging-connectivity probe script, exercising the new reference added for microsoft#2511. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * fix: address PR review comments on messaging connectivity probe - bash --help no longer prints the shebang line (filter out #! lines) - PowerShell HTTPS check drops -SkipHttpErrorCheck (unavailable in Windows PowerShell 5.1) and extracts the HTTP status from the caught exception across both PS editions - service-troubleshooting.md links the scripts as markdown links and states the working directory (skill root) for the example commands - eval stimulus adds an earlyTerminate to cap cost and documents why area:output is intentional Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * fix: address PR review comments (Round 2) Remove [Parameter(Mandatory)] from the PowerShell script's Namespace param and replace it with an explicit empty check that exits 2, avoiding interactive prompts in non-interactive scenarios. Remove the global \Continue = 'Stop' so a stray non-terminating error no longer aborts the whole probe; the DNS/HTTPS/TCP blocks already use explicit -ErrorAction Stop inside try/catch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * test: assert agent attempts to run messaging connectivity probe Rework the azure-diagnostics connectivity-probe eval so it verifies the agent *attempts* to run test-messaging-connectivity (a shell tool call), rather than merely mentioning the script name in prose. Uses a tool-call-match early-terminate that fires on the attempt and the built-in tool-calls grader, mirroring the azure-validate pattern. No live namespace is required since we only observe the attempt, not the result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * test: make connectivity-probe eval drive the agent to run the script Validated locally against a live agent (5 runs, 93%). Two fixes from the local run: (1) reframe the prompt as a production-incident triage so it reliably routes to azure-diagnostics (5/5) instead of azure-messaging or no skill; (2) early-terminate on tool-call-result rather than tool-call-match, because vally's tool-calls grader only matches completed tool calls — terminating on the call start dropped the script's result before it could be graded. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * chore: relocate messaging probe scripts after upstream tree reorg Upstream/main moved plugin/ -> plugins/azure-skills/ and evals/<skill>/ -> evals/azure-skills/<skill>/. The merge relocated modified files via rename detection, but the two new probe scripts (pure additions) had to be moved manually. Also update stale plugin/skills/azure-diagnostics path text in service-troubleshooting.md and migrate the connectivity-probe eval to the newly-added shell-command-invoked grader (terminating on the tool-call start). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * test: pin azure-diagnostics via requiredSkills in probe eval The multi-plugin restructure (microsoft#2872) changed the vally skill-loading model: without a requiredSkills tag every plugin skill is loaded and competes for routing, so the behavior probe test dropped to 0/5 (agent hand-rolled its own DNS probe instead of loading azure-diagnostics). Pin azure-diagnostics so its description survives char-budget truncation, matching the sibling-eval convention. Back to 5/5 skill-invocation + shell-command-invoked. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c * fix: make messaging probe sh executable and invoke via bash in docs Address PR review: the committed .sh was mode 100644, so the documented ./scripts/test-messaging-connectivity.sh invocation would fail with a permission error on Linux/macOS after checkout. Set the executable bit (100755) and invoke the script through �ash in both bash examples so the quick connectivity test runs regardless of how the file mode survives the build copy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 08574dee-8dd0-4771-98ad-934a5553060c
* eval: use vally turns for multi-turn tests * address Copilot feedback * one more comment fix * convert remaining followUp tags * convert followUp for azure-deploy and azure-cloud-migrate
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ation (microsoft#2934) * feat(azure-diagnostics): add run-ig script for Inspektor Gadget invocation Replace the inline kubectl debug ... ig run command assembly in the IG reference with cross-platform run-ig.sh / run-ig.ps1 scripts. The scripts resolve the node from a pod, inject the pinned IG image/version, apply the gadget-type default timeout, add k8s filters, and handle the tcpdump variant. A --dry-run flag prints the assembled command. Update inspektor-gadget.md and the caller docs (pod-failures, networking, node-issues, command-flows, aks-troubleshooting) to reference the script instead of hand-built commands. Fixes microsoft#2508 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92c8ec36-b76b-4825-896f-02598b79135e * fix(azure-diagnostics): address PR review feedback on run-ig scripts - run-ig.ps1: drop mandatory -Gadget param (avoid interactive prompt in non-interactive use); validate explicitly and exit with a clear message. - run-ig.ps1: remove \Continue='Stop' so Write-Error no longer terminates; guard pod node-resolution against empty/failed kubectl output. - Both scripts: only show the '| tcpdump -nvr -' pipe in the displayed command when tcpdump is present, so --dry-run matches real behavior; note when raw pcap-ng is emitted instead. - run-ig.sh: make usage() print only the leading comment block (no script code). - Docs: reference scripts/run-ig.sh (or run-ig.ps1) explicitly instead of a bare run-ig; note PowerShell PascalCase parameter names. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92c8ec36-b76b-4825-896f-02598b79135e * test(azure-diagnostics): add eval exercising run-ig script invocation Adds an integration stimulus that drives the agent to invoke the run-ig Inspektor Gadget helper script and early-terminates on the tool-call-match the instant the script is invoked, so the privileged kubectl debug never executes. No live cluster required. Grades skill invocation plus a tool-calls check confirming the script was actually called. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92c8ec36-b76b-4825-896f-02598b79135e * test(azure-diagnostics): use tool-call-result earlyTerminate for run-ig eval Address review feedback: switch the run-ig stimulus earlyTerminate from tool-call-match to tool-call-result. Terminating on the completed tool call reliably records the invocation for the grader; with no cluster/kubectl in CI the underlying kubectl debug fails instantly and harmlessly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92c8ec36-b76b-4825-896f-02598b79135e --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92c8ec36-b76b-4825-896f-02598b79135e
* fix: switch to managed identity for dashboard * update one more reference to key
The azure-kubernetes eval suite has no scoped owner, so changes under evals/azure-skills/azure-kubernetes/ fall through to the default `*` owner (@microsoft/github-copilot-for-azure-writers). Because the org ruleset requires code owner review, any PR touching both the skill and its evals needs two separate approvals: an AKS owner for the skill files, plus a writers-team member for the eval files -- even when the eval change is a small fixture that ships with the skill. Mirrors the existing /evals/azure-skills/microsoft-foundry/ entry and uses the same owner list as the azure-kubernetes skill directory.
* feat: relocate deploy-to-aks content under azure-kubernetes Carry over the deploy workflow, references, knowledge packs, and templates from the closed PR microsoft#1827 fork branch into a nested sub-skill directory. Frontmatter, routing, and self-references are fixed in follow-up commits. Refs: microsoft#1827 * feat: rename sub-skill to azure-kubernetes-app-deploy Rewrite frontmatter name and description (folded YAML per tests/AGENTS.md) with explicit DO-NOT-USE routing to the azure-kubernetes parent and the automatic-readiness sibling. Refs: microsoft#1827 * fix: update managed-by label to new sub-skill name Refs: microsoft#1827 * fix: correct go Dockerfile comments to reference ENTRYPOINT The template uses ENTRYPOINT but two header comments still said CMD, the same inconsistency review flagged and fixed on the dotnet template. Refs: microsoft#1827 * feat: point azure-kubernetes to the app-deploy sub-skill Add a Related-skills entry and a disambiguation note so cluster-vs-app deploy intent routes to azure-kubernetes-app-deploy. Refs: microsoft#1827 * test: add app-deploy stimuli to azure-kubernetes eval suite Six Vally stimuli (routing, boundary, output, workspace-fixture) covering the app-deploy sub-skill, plus a minimal Express fixture app. Grades on the parent skill name; replaces the deprecated Jest suite from microsoft#1827. Refs: microsoft#1827 * fix: address Copilot review on app-deploy sub-skill - SKILL.md: use inline double-quoted description (the repo frontmatter validator rejects >- folded scalars); the earlier folded-YAML premise was incorrect for this repo. - configmap.yaml: make data an explicit empty map ({}) so Kubernetes accepts the ConfigMap (bare data: parses as null). - deploy.yml: exclude <image> from the angle-bracket placeholder check so the workflow reaches the image-substitution step instead of failing on the intentionally-retained <image> token. Refs: microsoft#1827 * Genericize app-deploy Dockerfile base image tags Replace pinned base-image versions with <LATEST_STABLE_*> placeholders resolved at generation time. Refs: microsoft#2696 * Add app-deploy base image policy reference Single source for base-image selection and <LATEST_STABLE_*> resolution; documents the Microsoft/Azure Linux option. Refs: microsoft#2696 * Slim Python knowledge packs to durable guidance Remove inline Dockerfiles and pinned versions; keep framework deltas (collectstatic, init-container migrations, asyncpg, writable paths). Add pointer table. Refs: microsoft#2696 * Slim Node knowledge packs to durable guidance Remove inline Dockerfiles and pinned versions; keep framework deltas (Fastify 0.0.0.0 bind, Next standalone + ISR cache mount, Nest shutdown hooks). Add pointer table. Refs: microsoft#2696 * Slim compiled-language knowledge packs to durable guidance Remove inline Dockerfiles and pinned versions; keep framework deltas (Spring startupProbe, Go CGO/distroless + graceful shutdown, ASP.NET Data Protection keys). Add pointer table. Refs: microsoft#2696 * Reference canonical safeguards spec from app-deploy Point to the sibling constraint spec as source of truth; keep only the deploy-time auto-fix checklist. Refs: microsoft#2696 * Add base-image resolution step to quick-deploy Resolve <LATEST_STABLE_*> at generation time; clarify DS009-safe major-tag pinning; trim duplicated pack content. Refs: microsoft#2696 * Restore deploy-time image note in quick-deploy Re-add the deployment.yaml image-tag timing note and the RBAC admin-create alternative trimmed during token reduction. Refs: microsoft#2696 * Trim app-deploy SKILL.md under token limit Move detail into references; add base-images.md to the reference list. Refs: microsoft#2696 * Correct safeguards spec path and dotnet comment pin Fix the relative path to the sibling constraint spec (../ -> ../../) and de-pin a residual runtime-deps version in a dotnet template comment. Refs: microsoft#2696 * De-pin Gateway API CRD install to latest stable Replace the pinned gateway-api v1.0.0 release URL with the releases/latest/download form so the skill installs current stable CRDs at runtime instead of an Oct-2023 version. Refs: microsoft#2696 * Document distroless runtime placeholder exception Note in base-images.md why Go/Rust runtime stages keep the literal gcr.io/distroless/*-debian12 names instead of a <LATEST_STABLE_*> placeholder, so the build/runtime asymmetry is not mistaken for a bug. Refs: microsoft#2696
Replace vaibbavis with vaibbavisk20.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* feature: add azure-kusto-graph-skills plugin New plugin with 3 skills for Kusto graph analysis and IRQL security hunting: - azure-kusto-graph: KQL graph operators (make-graph, graph-match, shortest paths, components, persistent models) - azure-kusto-irql: composable IRQL pipelines (Get_*, Extract_*, Enrich_*) - azure-kusto-irql-graph: graph visualization (Lift_To_Graph, Graph_Render_View, node folding) Includes Vally eval routing tests for all 3 skills. Closes microsoft#3013, microsoft#3014, microsoft#3015 * fix: add #connect directive to default KQL output in all 3 skills * fix: commented-out #connect, auto-generate ADX Web Explorer version with graph-to-table * fix: Step 1/Step 2 labels, KC7 example #connect, ADX graph-to-table fallback * fix: add missing copilot-hooks.json referenced by plugin manifest * fix: address PR review comments — MCP server, skills.json, stray backticks, table separators, routing text, hooks ref * fix: remove hooks from plugin source — build copies shared hooks at build time * fix: add hooks references to all plugin manifests for telemetry * fix: address Kusto graph plugin CI validation --------- Co-authored-by: skill-terrain <skill-terrain@users.noreply.github.com>
…rosoft#3050) * Initial plan * fix: add eval CODEOWNERS entries for azure skills Co-authored-by: saikoumudi <22682497+saikoumudi@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: saikoumudi <22682497+saikoumudi@users.noreply.github.com>
* chore: refactor skill filtering in integration tests * fix comment * combine char-budget and skill-loader reorganize branching add unit tests * error message serialization * fix lint
…undry skill's skill.md (microsoft#3041)
Owner
Author
|
@copilot resolve the merge conflicts in this pull request |
|
@johnbilliris I can’t apply these changes because the pull request’s branch is the repository’s default branch, which I’m not allowed to push to directly. You can ask me to create a new pull request with the requested changes instead and I'll open a separate PR on a new branch. |
There was a problem hiding this comment.
Pull request overview
This PR is primarily a merge-from-main that brings in multi-plugin layout updates, test harness changes for multi-turn Vally stimuli, new/updated skills content (notably Microsoft Foundry wording updates and new AKS app-deploy material), plus build/manifest/hook plumbing across plugins.
Changes:
- Update Vally/agent-runner plumbing to support multi-turn stimuli via
stimulus.turnsand consolidate skill loading + char-budget logic undertests/utils/skill-loader.ts. - Introduce plugin hook manifest files and build logic to copy shared
hooks/into each built plugin output; add aplugin:newbootstrap script. - Add/expand skill content (Microsoft Foundry terminology updates, new
azure-kubernetes-app-deployskill assets) and add a newazure-kusto-graph-skillsplugin scaffold.
Reviewed changes
Copilot reviewed 249 out of 265 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/vally/vally-executor.ts | Adds multi-turn stimulus support and passes follow-up turns into AgentRunConfig. |
| tests/vally/tag-helpers.ts | Removes tag-based follow-up extraction helper. |
| tests/utils/char-budget.ts | Removes standalone char-budget helper module (moved into skill-loader). |
| tests/utils/agent-runner.ts | Refactors skill loading/disable logic to getSkillsForTest and updates session skill directory wiring. |
| tests/utils/tests/skill-loader.test.ts | Adds tests for truncateSkills and getSkillsForTest. |
| tests/utils/tests/char-budget.test.ts | Removes tests for deleted char-budget.ts. |
| tests/skills.json | Adds azure-kusto-graph-skills plugin listing + schedule. |
| tests/run-vally-test.ts | Removes result metadata field for follow-ups. |
| tests/microsoft-foundry/triggers.test.ts | Updates trigger prompts from “Azure AI Foundry” to “Microsoft Foundry”. |
| tests/microsoft-foundry/resource/create/triggers.test.ts | Updates trigger prompts terminology to “Microsoft Foundry”. |
| tests/microsoft-foundry/resource/create/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/models/deploy/deploy-model/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/models/deploy/customize-deployment/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/models/deploy/capacity/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/integration.test.ts | Updates integration prompt wording to “Microsoft Foundry”. |
| tests/microsoft-foundry/foundry-agent/trace/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/foundry-agent/observe/triggers.test.ts | Updates trigger prompt wording to “Microsoft Foundry”. |
| tests/microsoft-foundry/foundry-agent/observe/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/foundry-agent/eval-datasets/triggers.test.ts | Updates trigger prompt wording to “Microsoft Foundry”. |
| tests/microsoft-foundry/foundry-agent/eval-datasets/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/finetuning/triggers.test.ts | Updates trigger prompt wording to “Microsoft Foundry”. |
| tests/microsoft-foundry/finetuning/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/microsoft-foundry/snapshots/triggers.test.ts.snap | Snapshot updates (adds extracted keyword “authentication”). |
| tests/jest.config.ts | Updates coverage globs to include all plugin outputs under output/*/skills/**. |
| tests/azure-cloud-migrate/snapshots/triggers.test.ts.snap | Removes outdated snapshot. |
| tests/AGENTS.md | Updates skill path guidance from plugin/skills to plugins/{plugin}/skills. |
| tests/.npmrc | Switches npm registry setting. |
| scripts/src/vally/validate-stimulus.ts | Removes followUp tag validation. |
| scripts/src/tokens/tests/utils.test.ts | Updates token-limit override paths for new plugins layout. |
| scripts/src/tokens/tests/types.test.ts | Updates pattern-matching tests for new plugins layout. |
| scripts/src/shared/skill-helper.ts | Updates documentation strings for new plugins/<plugin>/skills structure. |
| scripts/src/plugin/bootstrap.ts | Adds a plugin scaffolding script invoked via npm script. |
| scripts/src/dashboard/tests/collectors/references.test.ts | Updates expected source paths to new plugin layout. |
| scripts/package.json | Adds plugin:new script entry. |
| scripts/.npmrc | Switches npm registry setting. |
| README.md | Updates hooks manifest references in the client capability table. |
| plugins/azure-skills/skills/microsoft-foundry/scripts/check-and-setup-dependencies.sh | Adds dependency check + azd extension install helper (bash). |
| plugins/azure-skills/skills/microsoft-foundry/scripts/check-and-setup-dependencies.ps1 | Adds dependency check + azd extension install helper (PowerShell). |
| plugins/azure-skills/skills/microsoft-foundry/references/standard-agent-setup.md | Terminology update to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/rbac/rbac.md | Updates RBAC link label to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/quota/references/optimization.md | Terminology update to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/quota/references/capacity-planning.md | Terminology updates + portal wording updates. |
| plugins/azure-skills/skills/microsoft-foundry/quota/quota.md | Terminology update to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/project/create/create-foundry-project.md | Terminology updates + header update. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/SKILL.md | Terminology update to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/scripts/generate_deployment_url.sh | Terminology update in script comments/help text. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/scripts/generate_deployment_url.ps1 | Terminology update in script comments. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/preset/SKILL.md | Terminology updates in prerequisites + related skills. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/preset/references/preset-workflow.md | Terminology updates in workflow prose + output text. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/preset/EXAMPLES.md | Terminology update in examples. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/customize/SKILL.md | Terminology updates in prerequisites + related skills. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/customize/references/customize-workflow.md | Terminology update for portal link mention. |
| plugins/azure-skills/skills/microsoft-foundry/models/deploy-model/capacity/scripts/discover_and_rank.ps1 | Terminology update in comment. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/troubleshoot/troubleshoot.md | Terminology update for endpoint label. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md | Updates link label to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/observe/references/cicd-monitoring.md | Updates link label to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/invoke/invoke.md | Expands protocol routing table and adds Activity protocol guidance. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/deploy/deploy.md | Clarifies deploy verification steps and Activity protocol post-deploy handling. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/scripts/check-copilot-app-entry.sh | Adds Copilot app entry preflight for plugin add-on installation. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/quick-start-hosted.md | Clarifies protocol support scope and deploy prompting. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/create-prompt.md | Adds explicit 403/RBAC row to error table. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/create/create-hosted.md | Updates local-run guidance and adds sample selection fallback guidance. |
| plugins/azure-skills/skills/microsoft-foundry/foundry-agent/azd-guidance/references/azd-ai-cli.md | Updates CLI quick reference and protocol list (adds activity). |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/workflows/quickstart.md | Terminology update to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/workflows/full-pipeline.md | Terminology update to “Microsoft Foundry”. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/SKILL.md | Terminology update in description + header. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/scripts/validate/validate_sft.py | Terminology update in docstring. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/scripts/validate/validate_rft.py | Terminology update in docstring + argparse description. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/scripts/validate/validate_dpo.py | Terminology update in docstring. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/scripts/submit_training.py | Terminology update in header/help text. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/scripts/deploy_model.py | Terminology update in header/help text. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/scripts/common.py | Terminology update in module/function docstrings. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/references/training-types.md | Terminology updates + portal wording update. |
| plugins/azure-skills/skills/microsoft-foundry/finetuning/references/agentic-rft.md | Terminology update in warning callout. |
| plugins/azure-skills/skills/azure-validate/SKILL.md | Replaces manual step table with workflow script instructions. |
| plugins/azure-skills/skills/azure-kubernetes/SKILL.md | Adds related-skill reference and clarifies when to use app-deploy sub-skill. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/mermaid/summary-dashboard.md | Adds deployment summary template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/mermaid/architecture-diagram.md | Adds mermaid architecture template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/serviceaccount.yaml | Adds ServiceAccount template w/ Workload Identity annotation. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/service.yaml | Adds ClusterIP Service template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/pdb.yaml | Adds PodDisruptionBudget template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/networkpolicy.yaml | Adds NetworkPolicy template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/namespace.yaml | Adds Namespace template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/ingress.yaml | Adds Ingress template for Web App Routing. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/httproute.yaml | Adds HTTPRoute template for Gateway API. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/hpa.yaml | Adds HPA template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/gateway.yaml | Adds Gateway template for Istio Gateway API. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/k8s/configmap.yaml | Adds ConfigMap template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/rust.dockerignore | Adds Rust dockerignore template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/rust.Dockerfile | Adds Rust Dockerfile template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/python.dockerignore | Adds Python dockerignore template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/python.Dockerfile | Adds Python Dockerfile template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/node.dockerignore | Adds Node dockerignore template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/node.Dockerfile | Adds Node Dockerfile template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/java.dockerignore | Adds Java dockerignore template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/go.dockerignore | Adds Go dockerignore template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/go.Dockerfile | Adds Go Dockerfile template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/dotnet.dockerignore | Adds .NET dockerignore template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/templates/dockerfiles/dotnet.Dockerfile | Adds .NET Dockerfile template. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/SKILL.md | Adds new azure-kubernetes-app-deploy skill definition. |
| plugins/azure-skills/skills/azure-kubernetes/azure-kubernetes-app-deploy/references/rollback.md | Adds rollback guidance reference. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/messaging/service-troubleshooting.md | Updates docs to prefer connectivity probe scripts. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/aks/references/command-flows.md | Updates AKS flow docs to prefer baseline script + run-ig wrapper. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/aks/pod-failures.md | Updates IG instructions to use run-ig script wrapper. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/aks/node-issues.md | Updates IG instructions to use run-ig wrapper. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/aks/networking.md | Updates IG instructions to use run-ig wrapper. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/aks/general-diagnostics.md | Updates baseline guidance to use aks-baseline script. |
| plugins/azure-skills/skills/azure-diagnostics/troubleshooting/aks/aks-troubleshooting.md | Updates fallback steps + baseline guidance to use scripts. |
| plugins/azure-skills/skills/azure-diagnostics/scripts/containerapp-diagnostics.ps1 | Adds Container Apps diagnostics collector (PowerShell). |
| plugins/azure-skills/skills/azure-diagnostics/scripts/appservice-diagnostics.ps1 | Adds App Service diagnostics collector (PowerShell). |
| plugins/azure-skills/skills/azure-diagnostics/references/container-apps/README.md | Updates docs to reference new diagnostics scripts. |
| plugins/azure-skills/skills/azure-diagnostics/references/app-service/README.md | Updates docs to reference new diagnostics scripts. |
| plugins/azure-skills/skills/azure-deploy/references/recipes/azd/scripts/apply-migrations.sh | Adds EF Core migrations script template for azd hook usage. |
| plugins/azure-skills/skills/azure-app-onboard/SKILL.md | Updates DO NOT USE FOR phrasing in description. |
| plugins/azure-skills/.plugin/plugin.json | Adds Copilot hooks reference in plugin manifest. |
| plugins/azure-kusto-graph-skills/version.json | Adds plugin-level NBGV version manifest. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-irql/version.json | Adds skill-level NBGV version manifest. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-irql/references/KUSTO_EXPLORER_LAUNCH.md | Adds Kusto Explorer launch procedure reference. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-irql/references/EXAMPLES.md | Adds usage examples reference. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-irql-graph/version.json | Adds skill-level NBGV version manifest. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-graph/version.json | Adds skill-level NBGV version manifest. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-graph/references/KUSTO_EXPLORER_LAUNCH.md | Adds Kusto Explorer launch procedure reference. |
| plugins/azure-kusto-graph-skills/skills/azure-kusto-graph/references/EXAMPLES.md | Adds usage examples reference. |
| plugins/azure-kusto-graph-skills/README.md | Adds plugin README. |
| plugins/azure-kusto-graph-skills/LICENSE | Adds MIT license for new plugin. |
| plugins/azure-kusto-graph-skills/.plugin/plugin.json | Adds Copilot plugin manifest for new plugin. |
| plugins/azure-kusto-graph-skills/.mcp.json | Adds MCP server config for new plugin. |
| plugins/azure-kusto-graph-skills/.cursor-plugin/plugin.json | Adds Cursor plugin manifest for new plugin. |
| plugins/azure-kusto-graph-skills/.claude-plugin/plugin.json | Adds Claude plugin manifest for new plugin. |
| package.json | Adds root plugin:new script passthrough. |
| hooks/cursor-hooks.json | Adds Cursor hooks manifest. |
| hooks/copilot-hooks.json | Adds Copilot/VS Code hooks manifest. |
| hooks/claude-hooks.json | Adds Claude hooks manifest. |
| gulpfile.ts | Adds build step to copy shared hooks/ into per-plugin output. |
| evals/azure-skills/microsoft-foundry/fixture/openai-agents-sdk/requirements.txt | Adds fixture dependencies. |
| evals/azure-skills/microsoft-foundry/fixture/openai-agents-sdk/mcp_server.py | Adds fixture MCP server implementation. |
| evals/azure-skills/microsoft-foundry/fixture/openai-agents-sdk/Dockerfile | Adds fixture Dockerfile. |
| evals/azure-skills/microsoft-foundry/fixture/openai-agents-sdk/agent.py | Adds fixture agent service implementation. |
| evals/azure-skills/microsoft-foundry/fixture/openai-agents-sdk/.env.example | Adds fixture env example. |
| evals/azure-skills/microsoft-foundry/fixture/openai-agents-sdk/.dockerignore | Adds fixture dockerignore. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/requirements.txt | Adds fixture dependencies. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/mcp_server.py | Adds fixture MCP server implementation. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/main.py | Adds fixture app for Google ADK. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/Dockerfile | Adds fixture Dockerfile. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/customer_support_agent/agent.py | Adds fixture agent wiring for Google ADK. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/customer_support_agent/init.py | Adds fixture package init. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/.env.example | Adds fixture env example. |
| evals/azure-skills/microsoft-foundry/fixture/google-adk/.dockerignore | Adds fixture dockerignore. |
| evals/azure-skills/microsoft-foundry/fixture/claude-agent-sdk/requirements.txt | Adds fixture dependencies. |
| evals/azure-skills/microsoft-foundry/fixture/claude-agent-sdk/mcp_server.py | Adds fixture MCP server implementation. |
| evals/azure-skills/microsoft-foundry/fixture/claude-agent-sdk/Dockerfile | Adds fixture Dockerfile. |
| evals/azure-skills/microsoft-foundry/fixture/claude-agent-sdk/agent.py | Adds fixture agent service implementation. |
| evals/azure-skills/microsoft-foundry/fixture/claude-agent-sdk/.env.example | Adds fixture env example. |
| evals/azure-skills/microsoft-foundry/fixture/claude-agent-sdk/.dockerignore | Adds fixture dockerignore. |
| evals/azure-skills/microsoft-foundry/fixture/.gitignore | Adds fixture gitignore. |
| evals/azure-skills/microsoft-foundry/eval.yaml | Updates prompts to “Microsoft Foundry” and adds migration stimulus; adds new stimulus turns fields. |
| evals/azure-skills/azure-resource-visualizer/eval.yaml | Converts follow-up tag into turns list for multi-turn stimuli. |
| evals/azure-skills/azure-kubernetes/fixture/app/server.js | Adds Express fixture app. |
| evals/azure-skills/azure-kubernetes/fixture/app/package.json | Adds Express fixture package.json. |
| evals/azure-app-onboard/seeded-deploy.eval.yaml | Converts follow-up tag into turns list for multi-turn stimuli. |
| evals/azure-app-onboard/e2e-container-apps.eval.yaml | Converts follow-up tag into turns list for multi-turn stimuli. |
| evals/azure-app-onboard/e2e-appservice-free.eval.yaml | Converts follow-up tag into turns list for multi-turn stimuli. |
| evals/azure-app-onboard/e2e-appservice-depth.eval.yaml | Converts follow-up tag into turns list for multi-turn stimuli. |
| docs/VERSIONING.md | Updates documentation for multi-plugin layout. |
| docs/plugin-manifest.md | Adds documentation for plugin manifests. |
| docs/hooks.md | Adds documentation for hooks manifests and per-client differences. |
| dashboard/src/skills/App.tsx | Updates skill source-link mapping for multi-plugin output paths. |
| dashboard/src/skills/tests/skillLink.test.ts | Updates tests for new output path mapping. |
| dashboard/infra/modules/sync-function-app.bicep | Updates deployment storage auth to use user-assigned identity and adds RBAC role assignment. |
| dashboard/infra/modules/function-app.bicep | Updates deployment storage auth to use user-assigned identity and adds RBAC role assignment. |
| dashboard/infra/main.bicep | Wires principalId outputs into function modules. |
| .npmrc | Switches npm registry setting. |
| .github/workflows/test-azure-deploy.yml | Updates integration test invocation to test:vally with plugin/skill/tag args. |
| .github/workflows/test-all-integration.yml | Updates matrix skill naming and token/tool usage upload env var handling. |
| .github/skills/skill-reviewer/SKILL.md | Updates referenced skill paths for new plugins layout. |
| .github/skills/skill-reviewer/references/review-checklist.md | Updates service skill path reference to new plugins layout. |
| .github/skills/skill-authoring/references/token-budgets.md | Updates token-check path example to new plugins layout. |
| .github/skills/sensei/SKILL.md | Updates skill path references to new plugins layout. |
| .github/skills/investigate-integration-test/SKILL.md | Updates path reference to plugins/*/skills. |
| .github/ISSUE_TEMPLATE/skill_extension_request.yml | Updates skill path hint to plugins/*/skills. |
| .github/instructions/skill-files.instructions.md | Updates applyTo glob to new plugins layout. |
| .github/dependabot.yml | Removes dependabot cooldown entries. |
| .github/copilot-instructions.md | Updates skill scaffold path guidance and adds shell script executable check mention. |
| .github/CODEOWNERS | Adds additional owners and eval owners entries. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+136
to
138
| name: 'AzureWebJobsStorage__clientId' | ||
| value: userAssignedIdentityId | ||
| } |
Comment on lines
126
to
129
| { | ||
| name: 'DEPLOYMENT_STORAGE_CONNECTION_STRING' | ||
| value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${storageAccount.listKeys().keys[0].value}' | ||
| name: 'AzureWebJobsStorage__clientId' | ||
| value: userAssignedIdentityId | ||
| } |
Comment on lines
+8
to
+10
| function main() { | ||
| const pluginName = "new-plugin"; | ||
| const pluginManifestBase = { |
Comment on lines
+44
to
+46
| // Plugin root | ||
| fs.mkdirSync(pluginRoot); | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Merge main into branch
Checklist
N/A
Related Issues