Add azure-reliability skill (Functions) - #2241
Merged
Sai Koumudi Kaluvakolanu (saikoumudi) merged 24 commits intoMay 13, 2026
Merged
Conversation
Adds a new azure-reliability skill that assesses Azure PaaS apps (Functions, Container Apps, App Service) for zone redundancy, storage replication, multi-region, and health probes. Reports findings as an enabled/disabled checklist (no numeric scoring) and supports both live (CLI) and IaC (Bicep/Terraform) remediation paths.
…ult SKU, deploy order) - Note AVM module param naming differs from raw Bicep (skuName vs sku.name); detect with Select-String and patch the actual param in use. - Annotate FC1/Consumption health probe as 'code-only fix' in the checklist and risk table; do not patch healthCheckPath in IaC for these plans. - Switch all 'az graph query' examples to '--query data[] -o json' (table output only shows summary cols). - Handle no-SKU storage case: ARM/AVM defaults to Standard_GRS; add explicit sku/skuName instead of find-and-replace. - Recommend splitting deploys: safe patches first, then storage migration, then storage SKU patch (a failed redundancy update can fail the whole deploy).
…gger on FC1/Consumption Adding a /api/health endpoint on Flex Consumption / Consumption Function Apps means modifying app source code (new HTTP trigger), not a Bicep/Terraform patch. Add an explicit STOP gate in configure-health-probes.md, the SKILL.md risk table, and both IaC patching references so the agent must ask the user before touching code, and respects 'no' by leaving everything unchanged.
…and multi-region
Workflow now sequences remediation as: easy ZR + health probes first (CLI or 'Deploy 1' for IaC), then explicitly ASK the user before kicking off the slow storage migration ('to be fully ZR you also need ZRS storage - want to do it?'). After the storage step (or skip), re-assess, then a new Configuration Workflow Step 3 asks about multi-region failover and waits for yes/no/later before generating any Front Door IaC. Phase 3 UX note now defers to Step 3 instead of duplicating the offer.
Replace per-resource checklist (with mostly n/a cells and mixed symbols) with a feature-pivoted table: 4 rows (Zone redundancy compute, Zone-redundant storage, Health probes, Multi-region failover), each with a single status (🟢 ON / 🟡 PARTIAL / 🔴 OFF or storage SKU) and a bullet list of relevant resources with inline reasons. Drops n/a noise; reasons sit on the resource line where users can see them. Re-Assess uses the same format with 'now ON' / 'still off' annotations. Step 3 prompt text aligned to use 🟢 instead of ✅.
…ser to run them Path B Deploy 1, Storage migration, Deploy 2, and Multi-region all now run the deploy commands (azd up / az deployment / terraform apply) directly after a single yes/no confirmation, then continue to re-assess. Updates Skill Boundaries and Integration tables: 'Deploy IaC for reliability changes: Yes' (was 'No, hand off to azure-deploy'). Bicep and Terraform patching references updated to summarize the deploy plan and ask 'Ready for Deploy 1?' instead of giving the user a list of commands to run.
Removed duplications: - 'When to Use This Skill' section (duplicate of Skill Activation Triggers) - 'HARD STOPS' block (each stop already inline at the right step) - Best Practices items that restated the workflow (kept only the 2 unique tips) - Path B Step 2 'Deploy-order rule' callout (Steps 3-5 already detail the flow) - Phase 3 UX note about multi-region (reduced to a one-liner pointing to Step 3) - Skill Boundaries 'IMPORTANT' header (duplicated description / Quick Reference) Also: - Quick Reference now says 'Reliability assessment table' instead of 'Reliability Checklist' for consistency with Phase 3 terminology. - Storage SKU row in Step 2 risk table now points to the two-deploy flow inline. - Fixed literal \\u escape sequences in SKILL.md, iac-patching-bicep.md, iac-patching-terraform.md that leaked from a previous edit (\\u2014 -> em-dash, \\u2192 -> arrow, \\u26a0\\ufe0f -> warning, \\u2705 -> checkmark, etc).
…ervice,functions} before team handoff Container Apps: - Standardize 'az graph query' to use --query 'data[]' -o json (table mode hides projected fields) - Fix Configure: Health Probes example (was mixing --set-env-vars + --yaml -<<EOF heredoc, which conflicts and isn't pwsh-portable); use a probes.yaml file instead - Add AVM modules note pointing to br/public:avm/res/app/managed-environment and avm/res/app/container-app param naming - Add STOP gate before adding /health route to container code (consent required, mirrors Functions FC1 pattern) - Add Reporting section showing how each Container Apps resource maps to feature-pivoted assessment rows App Service: - Standardize 'az graph query' to use --query 'data[]' -o json - Add ARR affinity / clientAffinityEnabled query (sticky sessions break ZR + multi-region) - Add new 'Configure: Disable Client Affinity (ARR Affinity)' section above slots; multi-region note now references it - Add AVM modules note pointing to br/public:avm/res/web/serverfarm and avm/res/web/site param naming - Add Reporting section, including PARTIAL state for multi-region with affinity still enabled Functions: - Add Reporting section so Functions matches the new convention Deferred for the App Service / Container Apps teams to verify against current Microsoft docs: - P1v3 minimum capacity for ZR (table currently says 3; docs may now say 2) - Standard tier health check support (table claims yes; verify) - Auto-Heal, backup/restore, VNet integration HA notes (out of scope for this pass)
…c content Make each services/<svc>/reliability.md the single source of truth for that service's plan/SKU rules, assessment queries, CLI commands, IaC patches (Bicep + Terraform + AVM), and reporting hints. Strip duplicated content from shared references so they only contain platform-level mechanics: - zone-redundancy-checks.md: keep platform overview + cross-service all-in-one query + AZ regions list. Drop per-service queries and remediation (already in services/). - configure-zone-redundancy.md: become a thin pointer to per-service refs + storage prerequisite + verification command. - configure-health-probes.md: become a thin pointer to per-service refs + cross-service consent gate + best practices. - iac-patching-bicep.md / iac-patching-terraform.md: keep framework (When to Use, Detection, AVM modules note, deploy plan) and the single cross-service Storage patch. Per-service patches now live in services/. - health-probe-checks.md: keep Front Door / Traffic Manager / App Insights checks + best practices + Multi-region row reporting. Drop per-service queries. - SKILL.md Phase 2 reworded as 'platform discovery + per-service deep dive'. Path A and Path B Step 3 now point to per-service refs for compute commands/patches. Net -889 lines (no content lost; the deleted lines were already duplicated in services/). This sets up a clean 3-PR split: PR #1 = shared platform refs + services/functions/, PR microsoft#2 = services/app-service/, PR microsoft#3 = services/container-apps/.
…ed-env warning (Container Apps) Two pieces of unique safety content from the deleted shared files were not yet captured in the per-service files. Adding them back: - Functions: 'Consumption (Y1) - upgrade path required' subsection with Flex vs Premium tradeoffs and cost warning (was in deleted configure-zone-redundancy.md). - Container Apps: explicit STOP before deleting old environment, with the 'az containerapp list --environment' check command (was in deleted configure-zone-redundancy.md).
Madhura Bharadwaj (MadhuraBharadwaj-MSFT)
changed the title
This was referenced May 12, 2026
…s on description (69 -> 43 words) Tests: - unit.test.ts (33 tests): metadata, content sections, feature-pivoted output rules, no-numeric-scoring rule, staged remediation, deploy-execution claims, FC1 consent gate, references existence + Functions service file sections, frontmatter format - triggers.test.ts: 17 should-trigger prompts (assess, ZR, multi-region, storage, service-specific) + 8 should-not-trigger negatives (other clouds, unrelated tasks, other Azure skills) + edge cases + keyword snapshot - integration.test.ts: 4 skill-invocation rate tests (assessment, ZR enable, multi-region, storage) + 3 response-quality tests (asks for scope, staged-flow guidance, multi-region opt-in). Skipped without Copilot CLI auth. Sensei pass: - Description trimmed from 69 -> 43 words (Medium-only -> Medium-High compliance) - Length: 322 chars (within 150-1024 range) - WHEN: triggers preserved with quoted distinctive phrases - All three services (Functions / Container Apps / App Service) still mentioned - Local run: 33/33 unit + all triggers PASS
…nst functions-quickstart-javascript-azd Replaces the previous skill-invocation rate tests + behavior tests with a single end-to-end test that: 1. Clones Azure-Samples/functions-quickstart-javascript-azd into the agent workspace 2. Prompts 'Assess and improve reliability of my function app' 3. Asserts azure-reliability is invoked + assessment flags zone-redundancy gap 4. Steps through follow-ups: yes to ZR fix via IaC, yes to deploy, no to storage migration 5. Asserts re-assessment shows zone redundancy is now ON Documents why storage migration and multi-region are NOT exercised in CI: - Storage LRS->ZRS migration takes hours to days (live data conversion) - Multi-region deploy takes 10+ minutes + DNS propagation, costs ~2x compute Both are exercised manually before each demo / release; their consent gates are unit-tested. Requires az login + azd auth login to actually run; otherwise auto-skipped via SKIP_INTEGRATION_TESTS.
- Remove App Service and Container Apps rows from SKILL.md service tables (Phase 2 dispatch + Path A fix table) - those service files don't exist on this branch and the links would 404. They will be re-added by the companion PRs once they merge. - Add 'Additional References' section to services/functions/reliability.md linking to the official Microsoft Learn 'Reliability in Azure Functions' guide (Flex Consumption pivot). - Add CODEOWNERS entry: /plugin/skills/azure-reliability/ owned by @MadhuraBharadwaj-MSFT @saikoumudi.
First run timed out at 30m mid-azd-up (stuck in 'Initialize bicep provider' for 25m). Second run with 1h timeout completed successfully in ~52min: clone -> first azd up -> assess -> patch IaC for zone redundancy -> second azd up -> re-assess -> confirm zoneRedundant: true on the live plan.
Madhura Bharadwaj (MadhuraBharadwaj-MSFT)
marked this pull request as ready for review
May 12, 2026 18:12
Copilot started reviewing on behalf of
Madhura Bharadwaj (MadhuraBharadwaj-MSFT)
May 12, 2026 18:13
View session
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new azure-reliability skill under plugin/skills/ plus supporting references and a full set of unit/trigger/integration tests to assess and remediate reliability posture (zone redundancy, storage redundancy, health probes, multi-region) for Azure PaaS apps.
Changes:
- Introduces the
azure-reliabilityskill (SKILL.md + NBGV version config). - Adds shared reliability reference docs (ARG queries, staged remediation, IaC patching, multi-region setup).
- Adds unit/trigger/integration test coverage and trigger keyword snapshots for the new skill.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/azure-reliability/unit.test.ts | Unit tests asserting frontmatter/content structure, remediation gates, and required references. |
| tests/azure-reliability/triggers.test.ts | Trigger matcher tests for prompts that should/shouldn’t activate the skill. |
| tests/azure-reliability/integration.test.ts | End-to-end integration test driving a safe zone redundancy remediation flow on a sample Functions app. |
| tests/azure-reliability/fixtures/sample.json | Template fixture file scaffolded with the skill test folder. |
| tests/azure-reliability/snapshots/triggers.test.ts.snap | Snapshots for extracted trigger keywords and description-derived triggers. |
| plugin/skills/azure-reliability/version.json | NBGV per-skill version configuration for the new skill. |
| plugin/skills/azure-reliability/SKILL.md | Core skill instructions: assessment workflow, staged remediation, and consent gates. |
| plugin/skills/azure-reliability/references/zone-redundancy-checks.md | Platform-level zone redundancy discovery query + dispatch guidance. |
| plugin/skills/azure-reliability/references/storage-redundancy-checks.md | Storage redundancy concepts, ARG queries, and remediation guidance. |
| plugin/skills/azure-reliability/references/services/functions/reliability.md | Azure Functions-specific reliability rules, queries, CLI/IaC patches, and reporting hints. |
| plugin/skills/azure-reliability/references/multi-region-checks.md | ARG queries and criteria for detecting multi-region + global load balancer configuration. |
| plugin/skills/azure-reliability/references/iac-patching-terraform.md | Terraform patching framework and deploy plan (includes duplicated content to fix). |
| plugin/skills/azure-reliability/references/iac-patching-bicep.md | Bicep/AVM patching framework and deploy plan (includes duplicated content to fix). |
| plugin/skills/azure-reliability/references/health-probe-checks.md | Platform/global health probe checks (Front Door/TM/App Insights). |
| plugin/skills/azure-reliability/references/configure-zone-redundancy.md | Platform notes + verification for enabling zone redundancy (contains conflicting guidance to fix). |
| plugin/skills/azure-reliability/references/configure-storage.md | Storage upgrade/migration procedures (contains conflicting gate + a likely broken ARG output example). |
| plugin/skills/azure-reliability/references/configure-multi-region.md | Multi-region “how-to” including Bicep/Terraform/CLI examples (contains a hard-coded subscription GUID to replace). |
| plugin/skills/azure-reliability/references/configure-health-probes.md | Platform notes and consent gate for health probe configuration. |
| .github/CODEOWNERS | Adds codeowners for the new skill folder. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Addresses PR review feedback. CI (pr.yml) validates that every directory in output/skills/ is listed in tests/skills.json and covered by integrationTestSchedule. Adds azure-reliability to both: - skills array (alphabetical) - integrationTestSchedule under a dedicated 0 10 * * 2-6 slot, since the e2e (two azd up runs) takes ~52min and is comparable to azure-deploy which has its own slot.
Co-authored-by: Copilot <copilot@github.com>
- SKILL.md: collapse multi-line description to single string; list App Service / Container Apps explicitly as unsupported rows so dispatch is unambiguous.
- configure-zone-redundancy.md: clarify storage is part of the same fix (discover now, migrate later, after compute ZR); drop redundant App Service / Container Apps note (now in SKILL.md).
- configure-multi-region.md: remove hard-coded subscription ID from example.
- iac-patching-bicep.md / iac-patching-terraform.md: remove duplicated storage-SKU patch sections (already covered by configure-storage.md and the deploy-order rule above).
- tests/azure-reliability/integration.test.ts: fix stray underscore in suite name (${SKILL_NAME}_ -> ${SKILL_NAME}).
Sai Koumudi Kaluvakolanu (saikoumudi)
approved these changes
May 12, 2026
msalaman
approved these changes
May 13, 2026
Sai Koumudi Kaluvakolanu (saikoumudi)
merged commit May 13, 2026
ebd5f32
into
microsoft:main
12 checks passed
This was referenced May 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Introduces the
azure-reliabilityskill — assesses and remediates the reliability posture of Azure PaaS apps (zone redundancy, ZRS storage, health probes, multi-region failover) and can drive deploys end-to-end (azd up/terraform apply/az deployment) after user confirmation.📁 Browse the full skill folder
Scope of this PR
SKILL.md— assessment + configuration workflow, feature-pivoted assessment table, two-deploy storage flow, multi-region wait-and-confirm gatezone-redundancy-checks.md,storage-redundancy-checks.md,multi-region-checks.md,health-probe-checks.md,configure-*.md,iac-patching-{bicep,terraform}.md)services/functions/reliability.md— Functions-specific plan/SKU rules, assessment queries, CLI commands, IaC patches (Bicep + Terraform + AVM), reporting hintsCompanion PRs (drafts)
This is the first of three coordinated PRs for the same skill. The other two are open as drafts and will be marked ready once this PR merges:
azure-reliability/app-service(addsservices/app-service/reliability.md)azure-reliability/container-apps(addsservices/container-apps/reliability.md)The draft PRs currently show this PR's diff plus their own service file (because they're stacked on top of this branch). Once this PR merges into
main, their diffs auto-shrink to a single file each.Architecture note
Per-service content (plan/SKU tables, CLI commands, Bicep/Terraform/AVM patches, reporting hints) lives in
services/<service>/reliability.md. Shared references contain only platform-level mechanics (cross-service Resource Graph queries, storage SKU patches, Front Door / Traffic Manager probe checks, deploy-order rules). This keeps each service team in sole ownership of their own service file.Testing
Tested end-to-end against an Azure Functions Flex Consumption deployment in a personal subscription. The skill correctly identified ZR/storage/health-probe gaps, ran the staged remediation (compute ZR → storage migration → re-assess), and offered multi-region as an explicit follow-up.