Fix: azure-validate bypassed for containerized Container Apps deployments - #1796
Conversation
…f and add container-specific triggers to azure-validate - Add Container Apps/containerized keywords to azure-validate SKILL.md description - Change azure-prepare Phase 2 Step 7 from⚠️ to ⛔ MANDATORY with explicit no-azd-up warning - Strengthen azure-prepare Next section with ⛔ and explicit Container Apps coverage - Add Container Apps trigger tests to azure-validate/triggers.test.ts - Update snapshots for both skills - Bump azure-validate version 1.0.3 → 1.0.4 - Bump azure-prepare version 1.1.12 → 1.1.13 Agent-Logs-Url: https://github.com/microsoft/GitHub-Copilot-for-Azure/sessions/59762ee6-88b6-46bf-916f-fd793f66ba29 Co-authored-by: fanyang-mono <52458914+fanyang-mono@users.noreply.github.com>
Updated the description and guidance for the azure-validate skill, including validation steps and prerequisites.
Details# 🔍 Token Analysis Report
📊 Token Change ReportComparing Summary
Changed Files
📊 Token Limit Check ReportChecked: 546 files
|
| File | Tokens | Limit | Over By |
|---|---|---|---|
.github/skills/analyze-test-run/SKILL.md |
2471 | 500 | +1971 |
.github/skills/file-test-bug/SKILL.md |
628 | 500 | +128 |
.github/skills/sensei/README.md |
3531 | 2000 | +1531 |
.github/skills/sensei/SKILL.md |
3026 | 500 | +2526 |
.github/skills/sensei/references/EXAMPLES.md |
3701 | 2000 | +1701 |
.github/skills/sensei/references/LOOP.md |
4181 | 2000 | +2181 |
.github/skills/sensei/references/SCORING.md |
4299 | 2000 | +2299 |
.github/skills/skill-authoring/SKILL.md |
839 | 500 | +339 |
plugin/skills/appinsights-instrumentation/SKILL.md |
908 | 500 | +408 |
plugin/skills/azure-ai/SKILL.md |
817 | 500 | +317 |
plugin/skills/azure-aigateway/SKILL.md |
1258 | 500 | +758 |
plugin/skills/azure-aigateway/references/policies.md |
2342 | 2000 | +342 |
plugin/skills/azure-cloud-migrate/SKILL.md |
559 | 500 | +59 |
plugin/skills/azure-cloud-migrate/references/services/container-apps/cloudrun-deployment-guide.md |
2029 | 2000 | +29 |
plugin/skills/azure-cloud-migrate/references/services/functions/lambda-to-functions.md |
2600 | 2000 | +600 |
plugin/skills/azure-cloud-migrate/references/services/functions/runtimes/javascript.md |
2181 | 2000 | +181 |
plugin/skills/azure-compliance/SKILL.md |
1185 | 500 | +685 |
plugin/skills/azure-compute/SKILL.md |
755 | 500 | +255 |
plugin/skills/azure-compute/workflows/vm-recommender/vm-recommender.md |
2393 | 2000 | +393 |
plugin/skills/azure-cost/SKILL.md |
1861 | 500 | +1361 |
plugin/skills/azure-deploy/SKILL.md |
1643 | 500 | +1143 |
plugin/skills/azure-deploy/references/pre-deploy-checklist.md |
3224 | 2000 | +1224 |
plugin/skills/azure-deploy/references/recipes/azd/errors.md |
3452 | 2000 | +1452 |
plugin/skills/azure-deploy/references/troubleshooting.md |
2038 | 2000 | +38 |
plugin/skills/azure-diagnostics/SKILL.md |
1132 | 500 | +632 |
plugin/skills/azure-diagnostics/aks-troubleshooting/networking.md |
2147 | 2000 | +147 |
plugin/skills/azure-diagnostics/aks-troubleshooting/node-issues.md |
2003 | 2000 | +3 |
plugin/skills/azure-enterprise-infra-planner/SKILL.md |
999 | 500 | +499 |
plugin/skills/azure-enterprise-infra-planner/references/constraints/compute-apps.md |
2022 | 2000 | +22 |
plugin/skills/azure-hosted-copilot-sdk/SKILL.md |
1260 | 500 | +760 |
plugin/skills/azure-kubernetes/SKILL.md |
2266 | 500 | +1766 |
plugin/skills/azure-kusto/SKILL.md |
2149 | 500 | +1649 |
plugin/skills/azure-messaging/SKILL.md |
967 | 500 | +467 |
plugin/skills/azure-prepare/SKILL.md |
3197 | 500 | +2697 |
plugin/skills/azure-prepare/references/aspire.md |
4003 | 2000 | +2003 |
plugin/skills/azure-prepare/references/plan-template.md |
2559 | 2000 | +559 |
plugin/skills/azure-prepare/references/recipes/azd/aspire.md |
3069 | 2000 | +1069 |
plugin/skills/azure-prepare/references/recipes/azd/terraform.md |
3012 | 2000 | +1012 |
plugin/skills/azure-prepare/references/research.md |
2217 | 2000 | +217 |
plugin/skills/azure-prepare/references/resources-limits-quotas.md |
3322 | 2000 | +1322 |
plugin/skills/azure-prepare/references/security.md |
2147 | 2000 | +147 |
plugin/skills/azure-prepare/references/services/functions/bicep.md |
3065 | 2000 | +1065 |
plugin/skills/azure-prepare/references/services/functions/templates/SPEC-composable-templates.md |
6187 | 2000 | +4187 |
plugin/skills/azure-prepare/references/services/functions/templates/recipes/composition.md |
4649 | 2000 | +2649 |
plugin/skills/azure-prepare/references/services/functions/terraform.md |
3358 | 2000 | +1358 |
plugin/skills/azure-quotas/SKILL.md |
3445 | 500 | +2945 |
plugin/skills/azure-quotas/references/commands.md |
2644 | 2000 | +644 |
plugin/skills/azure-resource-lookup/SKILL.md |
1288 | 500 | +788 |
plugin/skills/azure-resource-visualizer/SKILL.md |
2054 | 500 | +1554 |
plugin/skills/azure-storage/SKILL.md |
1180 | 500 | +680 |
plugin/skills/azure-upgrade/SKILL.md |
1001 | 500 | +501 |
plugin/skills/azure-upgrade/references/services/functions/automation.md |
3463 | 2000 | +1463 |
plugin/skills/azure-upgrade/references/services/functions/consumption-to-flex.md |
2773 | 2000 | +773 |
plugin/skills/azure-validate/SKILL.md |
916 | 500 | +416 |
plugin/skills/entra-app-registration/SKILL.md |
2067 | 500 | +1567 |
plugin/skills/entra-app-registration/references/api-permissions.md |
2545 | 2000 | +545 |
plugin/skills/entra-app-registration/references/cli-commands.md |
2211 | 2000 | +211 |
plugin/skills/entra-app-registration/references/console-app-example.md |
2752 | 2000 | +752 |
plugin/skills/entra-app-registration/references/oauth-flows.md |
2375 | 2000 | +375 |
plugin/skills/microsoft-foundry/SKILL.md |
2870 | 500 | +2370 |
plugin/skills/microsoft-foundry/foundry-agent/create/create.md |
3016 | 2000 | +1016 |
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md |
5555 | 2000 | +3555 |
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md |
2342 | 2000 | +342 |
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md |
4268 | 2000 | +2268 |
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md |
2547 | 2000 | +547 |
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md |
2701 | 2000 | +701 |
plugin/skills/microsoft-foundry/foundry-agent/troubleshoot/troubleshoot.md |
2164 | 2000 | +164 |
plugin/skills/microsoft-foundry/models/deploy-model/SKILL.md |
1640 | 500 | +1140 |
plugin/skills/microsoft-foundry/models/deploy-model/capacity/SKILL.md |
1739 | 500 | +1239 |
plugin/skills/microsoft-foundry/models/deploy-model/customize/SKILL.md |
2235 | 500 | +1735 |
plugin/skills/microsoft-foundry/models/deploy-model/customize/references/customize-workflow.md |
3335 | 2000 | +1335 |
plugin/skills/microsoft-foundry/models/deploy-model/preset/SKILL.md |
1226 | 500 | +726 |
plugin/skills/microsoft-foundry/models/deploy-model/preset/references/preset-workflow.md |
5534 | 2000 | +3534 |
plugin/skills/microsoft-foundry/quota/quota.md |
2288 | 2000 | +288 |
plugin/skills/microsoft-foundry/quota/references/capacity-planning.md |
2080 | 2000 | +80 |
plugin/skills/microsoft-foundry/references/sdk/foundry-sdk-py.md |
2162 | 2000 | +162 |
Consider moving content to
references/subdirectories.
Automated token analysis. See skill authoring guidelines for best practices.
There was a problem hiding this comment.
Pull request overview
This PR aims to prevent the azure-validate step from being skipped in containerized Azure Container Apps scenarios by strengthening skill routing triggers and making the azure-prepare → azure-validate → azure-deploy handoff explicitly mandatory.
Changes:
- Expanded
azure-validatetrigger wording to better associate it with Container Apps validation scenarios (and bumped skill version). - Made the
azure-preparehandoff toazure-validateexplicitly mandatory and explicitly prohibited running deployment commands directly (and bumped skill version). - Added Container Apps/containerized prompts to
azure-validatetrigger tests and updated snapshots accordingly.
Show a summary per file
| File | Description |
|---|---|
| plugin/skills/azure-validate/SKILL.md | Updates skill description triggers for Container Apps validation + version bump. |
| plugin/skills/azure-prepare/SKILL.md | Strengthens mandatory handoff language to enforce prepare → validate → deploy chain + version bump. |
| tests/azure-validate/triggers.test.ts | Adds Container Apps/containerized prompts to ensure azure-validate triggers as expected. |
| tests/azure-validate/snapshots/triggers.test.ts.snap | Updates snapshot for azure-validate description/keywords after trigger wording change. |
| tests/azure-prepare/snapshots/triggers.test.ts.snap | Updates snapshot due to new azure-prepare content affecting extracted keywords. |
Copilot's findings
- Files reviewed: 5/5 changed files
- Comments generated: 1
Jon Gallant (jongio)
left a comment
There was a problem hiding this comment.
Clean fix - the keyword-based trigger matching in trigger-matcher.ts confirms that adding 'Container Apps deployment' to the description is sufficient (substring matching picks up 'container' and 'apps' keywords for all 5 new test scenarios). The handoff strengthening from advisory to mandatory with explicit azd up prohibition addresses the root cause well.
…ents (microsoft#1796) * Initial plan * Fix container apps validation bypass: strengthen azure-prepare handoff and add container-specific triggers to azure-validate - Add Container Apps/containerized keywords to azure-validate SKILL.md description - Change azure-prepare Phase 2 Step 7 from⚠️ to ⛔ MANDATORY with explicit no-azd-up warning - Strengthen azure-prepare Next section with ⛔ and explicit Container Apps coverage - Add Container Apps trigger tests to azure-validate/triggers.test.ts - Update snapshots for both skills - Bump azure-validate version 1.0.3 → 1.0.4 - Bump azure-prepare version 1.1.12 → 1.1.13 Agent-Logs-Url: https://github.com/microsoft/GitHub-Copilot-for-Azure/sessions/59762ee6-88b6-46bf-916f-fd793f66ba29 Co-authored-by: fanyang-mono <52458914+fanyang-mono@users.noreply.github.com> * Revise azure-validate skill documentation Updated the description and guidance for the azure-validate skill, including validation steps and prerequisites. * Fix snapshot --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: fanyang-mono <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Fan Yang <yangfan@microsoft.com>
The agent was skipping
azure-validateand runningazd updirectly afterazure-preparefor containerized Container Apps scenarios, breaking the requiredazure-prepare → azure-validate → azure-deploychain.Two contributing causes:
azure-validatedescription lacked container/Container Apps keywords, weakening its association with containerized scenarios in skill routingazure-preparePhase 2 Step 7 used⚠️(advisory) for the handoff — makingazure-validateappear optionalChanges
plugin/skills/azure-validate/SKILL.md(v1.0.3 → v1.0.4)WHEN:triggers to description:"validate containerized app","validate Container Apps deployment","validate containerized web application","container app readiness check", etc.plugin/skills/azure-prepare/SKILL.md(v1.1.12 → v1.1.13)⚠️ Hand Off→⛔ MANDATORY Hand Offwith explicit prohibition: "Do NOT runazd up,azd deploy, or any deployment command directly"## Nextsection: upgraded⚠️→⛔, added explicit callout covering all scenario types with no exceptions:tests/azure-validate/triggers.test.tsShould TriggersuiteWarning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
centralus-2.in.applicationinsights.azure.com/home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp /home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp server start(dns block)/home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp /home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp server start git b-Co�� --get-regexp --global ure/node_modules/.bin/gh(dns block)/home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp /home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp server start git conf�� submodules | head -n 10 r ure/node_modules/.bin/gh(dns block)https://api.github.com/copilot_internal/user/opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node /home/REDACTED/work/GitHub-Copilot-for-Azure/GitHub-Copilot-for-Azure/tests/node_modules/@github/copilot/index.js --headless --no-auto-update --log-level error --stdio(http block)/opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node /home/REDACTED/work/GitHub-Copilot-for-Azure/GitHub-Copilot-for-Azure/tests/node_modules/@github/copilot/index.js --headless --no-auto-update --log-level error --stdio cal/bin/git(http block)/opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node /home/REDACTED/work/GitHub-Copilot-for-Azure/GitHub-Copilot-for-Azure/tests/node_modules/@github/copilot/index.js --headless --no-auto-update --log-level error --stdio github.com(http block)westus-0.in.applicationinsights.azure.com/home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp /home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp server start(dns block)/home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp /home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp server start git b-Co�� --get-regexp --global ure/node_modules/.bin/gh(dns block)/home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp /home/REDACTED/.npm/_npx/6027471c64e54353/node_modules/@azure/mcp-linux-x64/dist/azmcp server start git conf�� submodules | head -n 10 r ure/node_modules/.bin/gh(dns block)If you need me to access, download, or install something from one of these locations, you can either: