feat: replace azure-diagnostics AKS baseline sweep with aks-baseline script - #2936
Merged
Tom Meschter (tmeschter) merged 9 commits intoAug 5, 2026
Conversation
Adds cross-platform aks-baseline.sh/.ps1 that runs the read-only AKS cluster baseline sweep once and prints a single labeled digest (provisioning state, node pools, activity log, node readiness, unhealthy pods, kube-system health, warning events). Replaces the duplicated inline command blocks in general-diagnostics.md, aks-troubleshooting.md, and command-flows.md with script references. Fixes microsoft#2506 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
Tom Meschter (tmeschter)
requested review from
Rick Winter (RickWinter) and
Sai Koumudi Kaluvakolanu (saikoumudi)
as code owners
July 21, 2026 20:25
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the azure-diagnostics skill’s AKS troubleshooting guidance to replace repeated “baseline sweep” command blocks with a reusable cross-platform script (aks-baseline) that produces a single labeled diagnostic digest.
Changes:
- Added new
aks-baseline.shandaks-baseline.ps1scripts underplugin/skills/azure-diagnostics/scripts/to run the read-only AKS + Kubernetes baseline sweep. - Updated AKS troubleshooting markdown to reference the script (with bash + PowerShell invocation examples) instead of duplicating multiple CLI commands.
- Consolidated both the “cluster baseline” and “kubernetes baseline” fallback flows to reuse the same script.
Show a summary per file
| File | Description |
|---|---|
| plugin/skills/azure-diagnostics/troubleshooting/aks/references/command-flows.md | Replaces duplicated CLI fallback blocks with aks-baseline script references and examples. |
| plugin/skills/azure-diagnostics/troubleshooting/aks/general-diagnostics.md | Points baseline guidance at aks-baseline and removes the repeated command sweep. |
| plugin/skills/azure-diagnostics/troubleshooting/aks/aks-troubleshooting.md | Updates “Safe Fallback Checks” to run aks-baseline and then deep-dive as needed. |
| plugin/skills/azure-diagnostics/scripts/aks-baseline.sh | New bash implementation of the baseline sweep. |
| plugin/skills/azure-diagnostics/scripts/aks-baseline.ps1 | New PowerShell implementation of the baseline sweep. |
Review details
- Files reviewed: 5/5 changed files
- Comments generated: 2
- Review effort level: Low
- sh: usage() accepts an exit code (--help exits 0); validate a value exists before consuming each option so a missing value no longer loops - sh: set pipefail inside the warning-events subshell so kubectl failures are surfaced by run() - ps1: drop mandatory params (avoids interactive prompts in non-interactive runs); explicitly validate ResourceGroup/Cluster and exit early with a clear message - ps1: remove redundant \Continue assignment (default is Continue) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
JasonYeMSFT (JasonYeMSFT)
previously approved these changes
Jul 23, 2026
Adds a response-quality stimulus to the azure-diagnostics eval suite that supplies concrete cluster identity and asserts (via the tool-calls grader) that the agent actually runs aks-baseline.sh/.ps1 rather than pasting the individual az/kubectl commands. No real cluster is required — each script step is guarded, so the sweep completes and emits its digest regardless. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
Switches the script-execution stimulus to earlyTerminate on tool-call-match so the run aborts the moment the aks-baseline shell tool call starts, before the script actually executes. The triggering start event is still recorded, so the tool-calls grader confirms the agent chose to run the script; drops the completed grader per the early-terminate convention. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
The eval early-terminates the moment the aks-baseline script tool call starts, so no tool_result is ever emitted. The built-in tool-calls grader only counts a required tool on its tool_result, so it could never match an early-terminated call. Its inline (?i) regex flag also crashed the custom executor's tag-helpers, which compile earlyTerminate patterns with plain new RegExp(). - Drop (?i) / anchors from the earlyTerminate patterns (JS RegExp has no inline flags), matching the azure-deploy convention. - Add a small custom tool-call-started grader that matches the tool-call start event, and register it in vally-graders.ts. - Switch the stimulus to the new grader and make the prompt directive so the agent routes straight to the script (no real cluster exists in the test subscription). Verified locally: all 3 graders pass (100%); the run terminates at the script tool-call start with no tool_result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
Replace the custom tool-call-started grader with the built-in tool-calls grader, matching the pattern used by the azure-validate e2e suite. The built-in grader only matches a tool call once its result is recorded, so switch earlyTerminate from tool-call-match (aborts at the call's start, before completion) to tool-call-result (fires once the call completes). The aks-baseline script is read-only and fully guarded, so against a missing cluster every step fails fast and the sweep still exits cleanly in seconds. The call completes -> its tool_result is recorded -> tool-calls equired matches; termination then stops the run before any follow-up turns. No custom grader needed. Verified locally: all 3 graders pass (100%); the script tool call has a matching tool_result and the run ends at 2 turns. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
…ostic-sweep-script
…te eval to requiredSkills The multi-plugin restructure (microsoft#2872) moved plugin content to plugins/azure-skills/ and changed the eval executor to load skills only for stimuli declaring a per-stimulus requiredSkills tag. This merge: - Moves the new aks-baseline.sh/.ps1 scripts, orphaned by the merge at the old plugin/ path, to plugins/azure-skills/skills/azure-diagnostics/scripts/. - Adds requiredSkills: [azure-diagnostics] to all azure-diagnostics eval stimuli so the skill is loaded into the agent (without it, the agent gets no skills and every run reports 'Skills used 0'). Verified: build, frontmatter, references, validate-stimulus, typecheck, lint all pass; the aks-baseline script eval passes 3/3 (100%). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
…l failures Address PR review: the phase-based field selector (status.phase!=Running,!=Succeeded) missed pods that stay in phase Running while a container fails — CrashLoopBackOff, ImagePullBackOff, readiness-probe failures, and high restart counts. Step 5 now lists all pods (kubectl get pods -A -o wide) and flags a pod when it is not Ready, has a non-Running/non-terminal STATUS, or has a high restart count. Terminal pods (Completed/Succeeded) are excluded so finished jobs are not falsely reported. Both the bash and PowerShell scripts use the same READY/STATUS/RESTARTS logic and were verified to flag identical rows. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 56abebc8-418a-4684-ae16-b22c86d83e72
JasonYeMSFT (JasonYeMSFT)
approved these changes
Jul 29, 2026
Sai Koumudi Kaluvakolanu (saikoumudi)
approved these changes
Aug 4, 2026
Tom Meschter (tmeschter)
deleted the
tmeschter-aks-diagnostic-sweep-script
branch
August 5, 2026 17:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Replaces the AKS cluster baseline diagnostic sweep in the
azure-diagnosticsskill with a reusable, cross-platform script, resolving #2506.Previously the same fixed, ordered, read-only "what's the state of my cluster" sweep (
az aks show,az aks nodepool list,az monitor activity-log list,kubectl get nodes/pods/events) was duplicated verbatim across three AKS files. The agent had to generate ~7 commands and paste ~7 raw output blobs on every run.Changes
plugin/skills/azure-diagnostics/scripts/aks-baseline.shandaks-baseline.ps1(co-located with the skill, matching theazure-quotasconvention). They run the read-only sweep once and print a single labeled digest: provisioning state, node pool summary, recent activity log, node readiness, unhealthy pods, kube-system health, and recent warning events, plus a closing summary. Params:-g/-n(required), optional--namespace/--subscription. Each step is guarded so a single failure (e.g.kubectlnot authenticated) prints a note and the sweep continues; both scripts are strictly read-only.general-diagnostics.md,aks-troubleshooting.md, andreferences/command-flows.mdto reference the script via a markdown link, briefly describe what it does, and show sample bash + PowerShell invocations, replacing the duplicated inline command blocks. Judgment prose (Evidence Order, Scope Buckets) and pod-specific deep-dive commands (kubectl describe,kubectl logs --previous) remain inline.Validation
referencesvalidator passes (no broken/escaped links, no orphaned files)npm run buildstamps versions and copies both scripts intooutput/skills/azure-diagnostics/scripts/bash -nand PowerShell parser both clean;.shstored with LF line endingsChecklist
cd tests && npm test) — no functional tests exercise these scripts; validated viareferences, token check, build, and script syntax checksfix:,feat:,feature:,chore:,misc:,test:,eval:Related Issues
Fixes #2506