feat: extract azure-diagnostics dump-everything blocks into scripts - #2935
Merged
Tom Meschter (tmeschter) merged 8 commits intoAug 6, 2026
Merged
Conversation
Replace the inline chained 'dump everything' diagnostic blocks in the container-apps and app-service references with maintained, cross-platform scripts (.sh + .ps1) under the skill's scripts/ folder. Scripts collect and label diagnostic sections (config, revisions/deployments, recent logs, etc.) and print a summary line; interpretation stays in agent prose. READMEs now link to the scripts with sample invocations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
The Azure CLI already defaults to JSON output, and the original inline block did not specify -o json. Removing it keeps the script's behavior identical to the original. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
Tom Meschter (tmeschter)
requested review from
Rick Winter (RickWinter) and
Sai Koumudi Kaluvakolanu (saikoumudi)
as code owners
July 21, 2026 20:24
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors the azure-diagnostics skill documentation to replace the previously inlined “dump everything” Azure CLI command blocks with maintained, cross-platform diagnostic scripts under the skill’s scripts/ folder.
Changes:
- Adds bash + PowerShell diagnostic scripts for Container Apps and App Service that collect and label relevant
azoutputs. - Updates the Container Apps and App Service reference docs to link to the new scripts and provide sample invocations.
- Preserves the “collect + label only” approach (no interpretation) while reducing inline command complexity in the references.
Show a summary per file
| File | Description |
|---|---|
| plugin/skills/azure-diagnostics/scripts/containerapp-diagnostics.sh | New bash script to collect Container App revisions/config/logs. |
| plugin/skills/azure-diagnostics/scripts/containerapp-diagnostics.ps1 | New PowerShell script to collect Container App revisions/config/logs. |
| plugin/skills/azure-diagnostics/scripts/appservice-diagnostics.sh | New bash script to collect App Service config/deployments/settings/domains. |
| plugin/skills/azure-diagnostics/scripts/appservice-diagnostics.ps1 | New PowerShell script to collect App Service config/deployments/settings/domains. |
| plugin/skills/azure-diagnostics/references/container-apps/README.md | Replaces inline combined block with links + examples for the new scripts. |
| plugin/skills/azure-diagnostics/references/app-service/README.md | Replaces inline combined block with links + examples for the new scripts. |
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 2
- Review effort level: Low
Bash: validate that a value follows each flag (clear error instead of set -u 'unbound variable') and reject unknown options instead of silently treating typos as positional args. PowerShell: drop [Parameter(Mandatory)] (which prompts interactively when omitted) in favor of an explicit check that errors and exits early in non-interactive scenarios. Addresses review comments on PR microsoft#2935. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
JasonYeMSFT (JasonYeMSFT)
previously approved these changes
Jul 23, 2026
Adds a vally response-quality eval (evals/azure-diagnostics/script-invocation.eval.yaml) with two stimuli that check the agent both routes to azure-diagnostics and asks to run the bundled containerapp-diagnostics / appservice-diagnostics script (via a tool-calls grader on the shell command). Mirrors the existing azure-quotas check-quota pattern. Does not require live Azure resources — it grades the script invocation, not its output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
The agent invokes the bundled diagnostic script but the run is aborted at that tool call, so the script is never actually executed. Drops the completed grader (forbidden alongside earlyTerminate). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
JasonYeMSFT (JasonYeMSFT)
previously approved these changes
Jul 23, 2026
Relocate diagnostics scripts and script-invocation eval into the new multi-plugin layout (plugins/azure-skills/, evals/azure-skills/). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
Per-stimulus runs override is a no-op today (microsoft/vally#430); move runs:1 to defaults and drop the ineffective per-stimulus overrides. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
JasonYeMSFT (JasonYeMSFT)
requested changes
Jul 29, 2026
The built-in tool-calls grader only counts a required match when it sees the tool_result (execution_complete). Terminating at tool-call-match (execution_start) left a dangling tool_call the grader could not match, so switch both stimuli to tool-call-result. The bundled scripts are read-only az queries that no-op without resources, so running them once is harmless and the run still aborts immediately after. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 32539c36-dd02-4771-af34-815fa2324edd
JasonYeMSFT (JasonYeMSFT)
approved these changes
Aug 5, 2026
Sai Koumudi Kaluvakolanu (saikoumudi)
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves #2509.
Replaces the two inline "dump everything" diagnostic blocks in the
azure-diagnosticsskill with maintained, cross-platform scripts.Previously both per-service references shipped a single inlined
bashblock chaining severalazcalls with&&and labeledechoseparators. These are now extracted into.sh+.ps1scripts under the skill'sscripts/folder, following the existingazure-quotasconvention.Changes
plugin/skills/azure-diagnostics/scripts/:containerapp-diagnostics.{sh,ps1}— labels & collects revisions, registry config, ingress config, recent logs (--tail 20)appservice-diagnostics.{sh,ps1}— labels & collects app config, recent deployments, app settings (names), custom domains--name/-Nameand--resource-group/-ResourceGroup(bash also accepts positional args and an optional--subscription)container-apps/README.mdandapp-service/README.mdnow link to the scripts (markdown links) with PowerShell and bash sample invocations instead of the inline blocks.Behavior
Script behavior is intentionally identical to the original inline blocks (same
azqueries, same--queryprojections). The Azure CLI defaults to JSON output, so no explicit-o jsonwas added.Validation
npm run build✅npm run frontmatter(built output) ✅npm run references✅ — 26 skills pass, scripts properly linked, no orphaned/escaped referencesnpm run tokens compare— noazure-diagnosticsregressions-n+ PowerShell parser syntax checks — all 4 scripts OKNote: the eval suite (
evals/azure-diagnostics/eval.yaml) and former integration tests only exercise skill routing/invocation, not script execution, so they're unaffected.