feat: replace azure-validate AZCLI/Bicep validation steps with a shared script - #2856
Merged
Tom Meschter (tmeschter) merged 2 commits intoJul 17, 2026
Merged
Conversation
…ed script Fixes microsoft#2503. Adds validate-deployment.{sh,ps1} (bash + PowerShell) that runs the fixed az validation sequence (CLI present, authenticated, bicep build, deployment validate, what-if) with per-step PASS/FAIL, a what-if change summary, and a non-zero exit on failure. Both the AZCLI and Bicep recipes now reference this single shared helper instead of duplicating the inline command steps. Adds a standalone-Bicep eval fixture and stimulus so coverage actually exercises the script. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a000b508-d45c-4d9e-85f9-394275303cac
Tom Meschter (tmeschter)
requested a review
from Rick Winter (RickWinter)
as a code owner
July 15, 2026 22:39
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors the azure-validate skill’s AZCLI + Bicep validation guidance by moving the repeated Azure CLI validation sequence into a shared, cross-platform helper script (bash + PowerShell), and updates eval coverage to ensure agents actually run the helper in a standalone Bicep (no azure.yaml) scenario.
Changes:
- Added shared validation helpers:
validate-deployment.shandvalidate-deployment.ps1implementing the ordered CLI/auth/build/validate/what-if checks with PASS/FAIL + OVERALL result. - Updated AZCLI and Bicep recipe docs to reference the shared helper instead of inline command sequences.
- Added a new standalone Bicep eval fixture and an E2E stimulus/grader requiring invocation of the new helper script.
Show a summary per file
| File | Description |
|---|---|
| plugin/skills/azure-validate/references/recipes/scripts/validate-deployment.sh | New bash helper for deterministic validation sequence + summary output. |
| plugin/skills/azure-validate/references/recipes/scripts/validate-deployment.ps1 | New PowerShell helper mirroring the bash helper behavior. |
| plugin/skills/azure-validate/references/recipes/bicep/README.md | Replaces inline validation steps with helper-script guidance and updates checklists. |
| plugin/skills/azure-validate/references/recipes/azcli/README.md | Replaces inline validation steps with helper-script guidance; keeps Docker/Policy steps. |
| evals/azure-validate/fixture/bicep-cli/infra/resources.bicep | New standalone Bicep fixture module used by the added E2E stimulus. |
| evals/azure-validate/fixture/bicep-cli/infra/main.parameters.json | New parameters file for standalone fixture (literal values, no azd substitution). |
| evals/azure-validate/fixture/bicep-cli/infra/main.bicep | New subscription-scope standalone Bicep entrypoint for the fixture. |
| evals/azure-validate/fixture/bicep-cli/index.html | New minimal app content for the standalone fixture. |
| evals/azure-validate/e2e-eval.yaml | Adds new E2E stimulus + graders requiring the validate-deployment script to be run. |
Review details
- Files reviewed: 9/9 changed files
- Comments generated: 6
- Review effort level: Low
Rick Winter (RickWinter)
previously approved these changes
Jul 16, 2026
Rick Winter (RickWinter)
left a comment
Rick Winter (RickWinter)
left a comment
Member
There was a problem hiding this comment.
This PR replaces duplicated Bicep and Azure CLI validation guidance with shared Bash and PowerShell helpers and adds an eval fixture. I found no additional findings beyond the existing inline discussion. Nothing additional blocks merge from this pass.
- sh: guard value-consuming options against missing values (exit 2) - sh: exclude shebang from --help output - sh: document exit code 2 in header comment - sh: use 'az account show --query name -o tsv' + exit code instead of grep/sed JSON parsing - ps1: remove redundant \ summary table; keep \ to drive exit code, set inline per step - ps1: document exit code 2; align auth check to --query/\0 - bicep README: include CLI presence in core-validation wording (checklist item + table row) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a000b508-d45c-4d9e-85f9-394275303cac
JasonYeMSFT (JasonYeMSFT)
approved these changes
Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2502.
Fixes #2503.
Replaces the inline, near-duplicate Azure CLI validation sequences in the
azure-validateskill's AZCLI and Bicep recipes with a single shared, cross-platform script.What changed
New shared script
plugin/skills/azure-validate/references/recipes/scripts/validate-deployment.{sh,ps1}(bash + PowerShell). It runs the fixed validation sequence in order and reports per-step PASS/FAIL:az version)az account show)az bicep build)az deployment {sub|group} validate)az deployment {sub|group} what-if) — with a concise Create/Modify/Delete change summaryFlags:
--scope sub|group,--location/--resource-group,--template(default./infra/main.bicep),--parameters(default./infra/main.parameters.json, auto-skipped if absent),--subscription. Prints anOVERALL: PASS/FAILline and exits non-zero on any failure. Both scripts are non-interactive/fail-fast (no blocking prompts), suitable for agent invocation.AZCLI recipe (
recipes/azcli/README.md): inline steps 1–5 replaced with a script-reference section (markdown links, explanation, sub & group examples, result interpretation). Docker build and Azure Policy steps retained.Bicep recipe (
recipes/bicep/README.md): inline steps 1–4 replaced with the same shared script reference; checklist table updated. Linting and Policy steps retained.Tests / evals
evals/azure-validate/fixture/bicep-cli/— a standalone Bicep project (noazure.yaml) that routesazure-prepareto the Bicep/AZCLI recipe (the one referencing the shared script). Uses literal parameter values soaz deployment validateworks without azd env substitution. Bicep compiles cleanly (az bicep build).evals/azure-validate/e2e-eval.yaml— "Runs validate-deployment Script - Standalone Bicep (az CLI)". Graders require theazure-validateskill to be invoked and the agent to actually runvalidate-deployment.(sh|ps1); they disallowazure-deploy,azd up|deploy, andaz deployment ... create.Validation performed
npm run build✅ (scripts copied to output),npm run references✅ (links stay within the skill), frontmatter ✅ on built output, token check shows no new failures.npm run vally validate-stimulus✅ (tags + earlyTerminate JSON valid).