Skip to content

feat: drive azure-validate steps via workflow.ps1 script - #2969

Merged
Tom Meschter (tmeschter) merged 5 commits into
microsoft:mainfrom
tmeschter:tmeschter-azure-validate-workflow-script
Aug 3, 2026
Merged

feat: drive azure-validate steps via workflow.ps1 script#2969
Tom Meschter (tmeschter) merged 5 commits into
microsoft:mainfrom
tmeschter:tmeschter-azure-validate-workflow-script

Conversation

@tmeschter

@tmeschter Tom Meschter (tmeschter) commented Jul 27, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #2972.

Replaces the inline nine-step ## Steps table in the azure-validate SKILL.md with a workflow driver script that walks the agent through each validation step one at a time. This puts the workflow "on rails" — leaving less room for the agent to skip or deviate from steps — and sets up incrementally making each step script-based for a more deterministic skill.

Changes

  • New references/scripts/workflow.ps1 — a step-driver script. The agent starts the workflow by calling it with -WorkspacePath (no -CompletedStep). Each response prints the next action plus the value to pass as -CompletedStep on the next call. The script records progress in .azure/validate-status.json itself, eliminating a per-step file-edit tool call by the agent.
  • New references/scripts/workflow.sh — a bash equivalent for macOS/Linux, producing byte-for-byte identical agent-facing output to the PowerShell version.
  • SKILL.md — the Steps section now instructs the agent to run the script (linking both the .ps1 and .sh variants) and follow its instructions until it reports the workflow is complete.

Review feedback addressed

  • Validate that -WorkspacePath / --workspace-path exists and is a directory, failing early with a clear message instead of a later, more confusing error.
  • Emit intended Markdown backticks in agent-facing output (they were previously consumed as PowerShell escapes); both scripts kept in sync.
  • Write .azure/validate-status.json as UTF-8 without a BOM so non-PowerShell JSON consumers can parse it (Windows PowerShell 5.1 defaulted to UTF-16LE).

Validation

  • npm run build succeeds (version stamping OK).
  • npm run references passes (both scripts linked from SKILL.md; no orphaned files).
  • Both scripts verified across all nine steps plus the start / invalid-step / missing-arg / bad-path cases; output is identical between the two.
  • azure-validate vally evals: routing 100%. E2E stimuli that reach azure-validate exercise the -CompletedStep flow cleanly with zero script errors. The remaining e2e variance is unrelated top-level routing (azure-app-onboard vs azure-prepareazure-validate) where azure-validate is never entered.

Replace the inline nine-step table in SKILL.md with a workflow.ps1 script that walks the agent through each validation step one at a time, tracking progress via completedStep in .azure/validate-status.json.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727
workflow.ps1 now accepts a -CompletedStep parameter and writes the progress to .azure/validate-status.json itself, instead of instructing the agent to edit the file. The agent starts the workflow by calling the script with no -CompletedStep, and each response tells it which value to pass next. This eliminates a per-step tool call by the agent. SKILL.md updated to document the new invocation pattern.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727
Copilot AI review requested due to automatic review settings July 27, 2026 19:34
@tmeschter
Tom Meschter (tmeschter) requested a review from a team as a code owner July 27, 2026 19:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the azure-validate skill to drive its validation process via a PowerShell step-driver script (workflow.ps1) instead of an inline multi-step table in SKILL.md, so an agent can advance through the workflow one step at a time while progress is persisted in the workspace.

Changes:

  • Replaced the inline “Steps” table in azure-validate/SKILL.md with instructions to run a workflow driver script repeatedly, passing -CompletedStep.
  • Added references/scripts/workflow.ps1 to emit the next action to perform and persist the last completed step to .azure/validate-status.json.
Show a summary per file
File Description
plugin/skills/azure-validate/SKILL.md Replaces the inline steps table with guidance to run workflow.ps1 iteratively using -CompletedStep.
plugin/skills/azure-validate/references/scripts/workflow.ps1 New PowerShell step-driver script that prints the next workflow action and writes .azure/validate-status.json.

Review details

Comments suppressed due to low confidence (2)

plugin/skills/azure-validate/references/scripts/workflow.ps1:67

  • Same PowerShell escaping issue here: backticks in double-quoted strings won’t be printed as intended. Switching to single quotes also requires escaping the apostrophe in "recipe's".
    Write-Output "Action: Copy the recipe's `Validation Steps` into `.azure/deployment-plan.md` as children of `All validation checks pass`."
    Write-Output "Next: re-run workflow.ps1 with -CompletedStep AddValidationSteps after completing the action."
    Write-Output "Reference: references/recipes/README.md, `.azure/deployment-plan.md"

plugin/skills/azure-validate/references/scripts/workflow.ps1:115

  • Backticks in these double-quoted strings are treated as PowerShell escapes, so inline-code formatting (and even the literal backticks) won’t appear in the output as written. Use single-quoted strings (or escape the backticks) for the intended output.
    Write-Output "Action: Only after ALL checks pass, set the plan status to `Validated`."
    Write-Output "Next: re-run workflow.ps1 with -CompletedStep UpdateStatus after completing the action."
    Write-Output "Reference: `.azure/deployment-plan.md"
  • Files reviewed: 2/2 changed files
  • Comments generated: 3
  • Review effort level: Low

Comment thread plugin/skills/azure-validate/references/scripts/workflow.ps1
Comment thread plugin/skills/azure-validate/references/scripts/workflow.ps1 Outdated
Comment thread plugin/skills/azure-validate/references/scripts/workflow.ps1 Outdated
Tom Meschter (tmeschter) and others added 3 commits July 27, 2026 14:10
Add a Mac/Linux bash equivalent of workflow.ps1 that drives the
azure-validate workflow step-by-step via --completed-step, mirroring
the PowerShell script's output exactly. Link both scripts (and restore
recipe/role reference links) from SKILL.md so orphan detection passes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727
- Validate the workspace path exists and is a directory, failing early
  with a clear message instead of a later New-Item/Set-Content error.
- Emit intended Markdown backticks in agent-facing output by using
  single-quoted PowerShell strings (backticks were being consumed as
  PowerShell escapes); mirror the same output in workflow.sh.
- Write validate-status.json as UTF-8 without BOM so non-PowerShell
  JSON consumers can parse it (Windows PowerShell 5.1 defaulted to
  UTF-16LE).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727
Upstream relocated the plugin tree from plugin/skills/ to
plugins/azure-skills/skills/. Resolved the resulting rename conflicts by
moving the azure-validate workflow.ps1 / workflow.sh scripts and the
SKILL.md Steps changes to the new location.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59ff3cb9-ce5c-4e7e-945b-5e597907a727
@tmeschter
Tom Meschter (tmeschter) merged commit 97cd69e into microsoft:main Aug 3, 2026
12 checks passed
@tmeschter
Tom Meschter (tmeschter) deleted the tmeschter-azure-validate-workflow-script branch August 3, 2026 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Make the top-level azure-validate workflow script-based

3 participants