Add cleanup script for test resources

[msalaman]

Add cleanup script to delete test resources

[github-actions]

Details

# 🔍 Token Analysis Report

@github-copilot-for-azure/scripts@1.0.0 tokens
node --import tsx src/tokens/cli.ts compare --base origin/main --head HEAD --markdown

fatal: path 'eng/README.md' exists on disk, but not in 'origin/main'

📊 Token Change Report

Comparing origin/main → HEAD

Summary

Metric	Value
📈 Total Change	+150 tokens (0%)
Before	0 tokens
After	150 tokens
Files Changed	1

Changed Files

File	Before	After	Change
eng/README.md	-	150	+150

@github-copilot-for-azure/scripts@1.0.0 tokens
node --import tsx src/tokens/cli.ts check --markdown

📊 Token Limit Check Report

Checked: 536 files
Exceeded: 71 files

⚠️ Files Exceeding Token Limits

File	Tokens	Limit	Over By
.github/skills/analyze-test-run/SKILL.md	2471	500	+1971
.github/skills/file-test-bug/SKILL.md	628	500	+128
.github/skills/sensei/README.md	3531	2000	+1531
.github/skills/sensei/SKILL.md	3026	500	+2526
.github/skills/sensei/references/EXAMPLES.md	3701	2000	+1701
.github/skills/sensei/references/LOOP.md	4181	2000	+2181
.github/skills/sensei/references/SCORING.md	4200	2000	+2200
.github/skills/skill-authoring/SKILL.md	839	500	+339
plugin/skills/appinsights-instrumentation/SKILL.md	908	500	+408
plugin/skills/azure-ai/SKILL.md	817	500	+317
plugin/skills/azure-aigateway/SKILL.md	1258	500	+758
plugin/skills/azure-aigateway/references/policies.md	2342	2000	+342
plugin/skills/azure-cloud-migrate/references/services/functions/lambda-to-functions.md	2600	2000	+600
plugin/skills/azure-cloud-migrate/references/services/functions/runtimes/javascript.md	2181	2000	+181
plugin/skills/azure-compliance/SKILL.md	1185	500	+685
plugin/skills/azure-compute/SKILL.md	755	500	+255
plugin/skills/azure-compute/workflows/vm-recommender/vm-recommender.md	2393	2000	+393
plugin/skills/azure-compute/workflows/vm-troubleshooter/references/cannot-connect-to-vm.md	7308	2000	+5308
plugin/skills/azure-cost/SKILL.md	1861	500	+1361
plugin/skills/azure-deploy/SKILL.md	1562	500	+1062
plugin/skills/azure-diagnostics/SKILL.md	1132	500	+632
plugin/skills/azure-diagnostics/aks-troubleshooting/networking.md	2147	2000	+147
plugin/skills/azure-diagnostics/aks-troubleshooting/node-issues.md	2003	2000	+3
plugin/skills/azure-enterprise-infra-planner/SKILL.md	991	500	+491
plugin/skills/azure-enterprise-infra-planner/references/constraints/compute-apps.md	2022	2000	+22
plugin/skills/azure-hosted-copilot-sdk/SKILL.md	778	500	+278
plugin/skills/azure-kubernetes/SKILL.md	2266	500	+1766
plugin/skills/azure-kusto/SKILL.md	2149	500	+1649
plugin/skills/azure-messaging/SKILL.md	967	500	+467
plugin/skills/azure-prepare/SKILL.md	2767	500	+2267
plugin/skills/azure-prepare/references/aspire.md	2991	2000	+991
plugin/skills/azure-prepare/references/plan-template.md	2559	2000	+559
plugin/skills/azure-prepare/references/recipes/azd/aspire.md	2101	2000	+101
plugin/skills/azure-prepare/references/recipes/azd/terraform.md	3012	2000	+1012
plugin/skills/azure-prepare/references/research.md	2217	2000	+217
plugin/skills/azure-prepare/references/resources-limits-quotas.md	3322	2000	+1322
plugin/skills/azure-prepare/references/security.md	2133	2000	+133
plugin/skills/azure-prepare/references/services/functions/bicep.md	3065	2000	+1065
plugin/skills/azure-prepare/references/services/functions/templates/SPEC-composable-templates.md	6187	2000	+4187
plugin/skills/azure-prepare/references/services/functions/templates/recipes/composition.md	4649	2000	+2649
plugin/skills/azure-prepare/references/services/functions/terraform.md	3358	2000	+1358
plugin/skills/azure-quotas/SKILL.md	3445	500	+2945
plugin/skills/azure-quotas/references/commands.md	2644	2000	+644
plugin/skills/azure-resource-lookup/SKILL.md	1288	500	+788
plugin/skills/azure-resource-visualizer/SKILL.md	2054	500	+1554
plugin/skills/azure-storage/SKILL.md	1180	500	+680
plugin/skills/azure-upgrade/SKILL.md	1001	500	+501
plugin/skills/azure-upgrade/references/services/functions/automation.md	3463	2000	+1463
plugin/skills/azure-upgrade/references/services/functions/consumption-to-flex.md	2773	2000	+773
plugin/skills/azure-validate/SKILL.md	906	500	+406
plugin/skills/entra-app-registration/SKILL.md	2067	500	+1567
plugin/skills/entra-app-registration/references/api-permissions.md	2545	2000	+545
plugin/skills/entra-app-registration/references/cli-commands.md	2211	2000	+211
plugin/skills/entra-app-registration/references/console-app-example.md	2752	2000	+752
plugin/skills/entra-app-registration/references/oauth-flows.md	2375	2000	+375
plugin/skills/microsoft-foundry/SKILL.md	2870	500	+2370
plugin/skills/microsoft-foundry/foundry-agent/create/create.md	3016	2000	+1016
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	5511	2000	+3511
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	2342	2000	+342
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	4268	2000	+2268
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	2547	2000	+547
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	2701	2000	+701
plugin/skills/microsoft-foundry/models/deploy-model/SKILL.md	1640	500	+1140
plugin/skills/microsoft-foundry/models/deploy-model/capacity/SKILL.md	1739	500	+1239
plugin/skills/microsoft-foundry/models/deploy-model/customize/SKILL.md	2235	500	+1735
plugin/skills/microsoft-foundry/models/deploy-model/customize/references/customize-workflow.md	3335	2000	+1335
plugin/skills/microsoft-foundry/models/deploy-model/preset/SKILL.md	1226	500	+726
plugin/skills/microsoft-foundry/models/deploy-model/preset/references/preset-workflow.md	5534	2000	+3534
plugin/skills/microsoft-foundry/quota/quota.md	2130	2000	+130
plugin/skills/microsoft-foundry/quota/references/capacity-planning.md	2080	2000	+80
plugin/skills/microsoft-foundry/references/sdk/foundry-sdk-py.md	2162	2000	+162

Consider moving content to references/ subdirectories.

---

Automated token analysis. See skill authoring guidelines for best practices.

[Copilot]

Pull request overview

Adds PowerShell tooling under eng/ to help clean up test Azure subscription resources, including helper modules for resource purging and GitHub↔MS alias metadata lookup.

Changes:

• Added test-sub-cleanup.ps1 to tag and/or delete non-compliant resource groups and optionally remove deployments.
• Added Resource-Helpers.ps1 to discover and purge soft-deleted/purgeable resources (Key Vault, Managed HSM, Cognitive Services) and handle storage cleanup edge-cases.
• Added Metadata-Helpers.ps1 plus a short eng/README.md with usage instructions.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 17 comments.

File	Description
eng/scripts/test-sub-cleanup.ps1	Main cleanup workflow: login, enumerate RGs, tag/delete, and subscription deployment cleanup.
eng/scripts/Resource-Helpers.ps1	Helper functions for purgeable resources and storage “WORM/immutability” cleanup.
eng/scripts/Metadata-Helpers.ps1	Helper functions to call repos.opensource.microsoft.com to build alias cache.
eng/README.md	Minimal run instructions for the cleanup script.

Comments suppressed due to low confidence (3)

eng/scripts/test-sub-cleanup.ps1:365

• PowerShell 7+ null-conditional operator is used here ($_.Outputs?.Count / $_.Parameters?.ContainsKey(...)). With #Requires -Version 6.0 this will be a parse-time failure on PS 6. Either update the required version or refactor to PS6-compatible null checks.

        $toDelete = @(Get-AzResourceGroupDeployment -ResourceGroupName $ResourceGroup.ResourceGroupName `
            | Where-Object { $_ -and ($_.Outputs?.Count -or $_.Parameters?.ContainsKey('testApplicationSecret')) })
    }

eng/scripts/test-sub-cleanup.ps1:517

• PowerShell 7+ null-conditional operator is used here ($rg.Tags?.ContainsKey(...)). With #Requires -Version 6.0 this will be a parse-time failure on PS 6. Either update the required version or refactor to PS6-compatible null checks.

                # For storage tests specifically, if they are aborted then blobs with immutability policies
                # can be left around which prevent deletion.
                if ($rg.Tags?.ContainsKey('ServiceDirectory') -and $rg.Tags.ServiceDirectory -like '*storage*') {
                    SetStorageNetworkAccessRules -ResourceGroupName $rg.ResourceGroupName -SetFirewall -CI:($null -ne $env:SYSTEM_TEAMPROJECTID)
                    Remove-WormStorageAccounts -GroupPrefix $rg.ResourceGroupName -CI:($null -ne $env:SYSTEM_TEAMPROJECTID)

eng/scripts/Resource-Helpers.ps1:382

• This uses the PowerShell 7+ ternary operator (condition ? a : b). If the cleanup tooling is intended to run on PowerShell 6 (as test-sub-cleanup.ps1 currently requires), this will fail to parse. Align the required PowerShell version across scripts or rewrite to PS6-compatible syntax.

function RemoveStorageAccount($Account) {
  Write-Host ($WhatIfPreference ? 'What if: ' : '') + "Readying $($Account.StorageAccountName) in $($Account.ResourceGroupName) for deletion"
  # If it doesn't have containers then we can skip the explicit clean-up of this storage account
  if ($Account.Kind -eq "FileStorage") { return }

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (1)

eng/scripts/Resource-Helpers.ps1:381

• Inside the per-storage-account loop, return exits SetStorageNetworkAccessRules entirely, so later storage accounts in the same resource group won't get updated. This should likely be continue (or refactor the condition) so the function can process all accounts.

        $ipRanges = Get-AzStorageAccountNetworkRuleSet -ResourceGroupName $ResourceGroupName -Name $account.Name
        if ($ipRanges) {
          foreach ($range in $ipRanges.IpRules) {
            if (DoesSubnetOverlap $range.IPAddressOrRange $clientIp) {
              return
            }

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

eng/scripts/test-sub-cleanup.ps1:636

• DeleteSubscriptionDeployments is called unconditionally, even though deployment deletion appears to be intended to be optional (there is a -DeleteArmDeployments switch, but it isn't used). Consider gating this call behind an explicit switch (or reusing -DeleteArmDeployments) so running cleanup doesn't always wipe subscription-scoped deployment history.

try {
    DeleteOrUpdateResourceGroups
    DeleteSubscriptionDeployments
}

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

eng/scripts/test-sub-cleanup.ps1:22

• In the Provisioner parameter set, ProvisionerApplicationSecret is optional, but the only provisioner login path requires it. If a user supplies the mandatory provisioner parameters without the secret (and without an existing Az context), the script will fail with an interactive-login oriented message even though -Login isn’t valid for this parameter set. Consider making the secret mandatory for Provisioner, or adding an alternative non-interactive auth option (and adjusting the error message accordingly).

    [Parameter(ParameterSetName = 'Provisioner', Mandatory = $false)]
    [string] $ProvisionerApplicationSecret,

eng/scripts/Resource-Helpers.ps1:217

• Spelling/grammar: the warning message says "Failed to deleted Managed HSM"; it should be "Failed to delete" (or "Failed to purge") to read correctly and avoid confusion in logs.

            if ($content.error) {
              $err = $content.error
              Write-Warning "Failed to deleted Managed HSM '$($r.Name)': ($($err.code)) $($err.message)"
            }

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

eng/scripts/test-sub-cleanup.ps1:642

• DeleteSubscriptionDeployments is executed on every non-dry run (regardless of -DeleteArmDeployments) and will remove all subscription-scoped deployments in the selected subscription. This is a broad/destructive side effect that should be opt-in (e.g., gated behind -DeleteArmDeployments or a dedicated switch) and ideally protected with ShouldProcess/confirmation.

try {
    DeleteOrUpdateResourceGroups
    if (-not $DryRun) {
        DeleteSubscriptionDeployments
    }

eng/scripts/test-sub-cleanup.ps1:416

• DeleteArmDeployments / DeleteSubscriptionDeployments perform deletion (Remove-*Deployment) without ShouldProcess checks even though the script/function cmdlets declare SupportsShouldProcess. Consider wrapping the removal calls in if ($Force -or $PSCmdlet.ShouldProcess(...)) so -WhatIf/confirmation semantics work consistently.

function DeleteArmDeployments([object]$ResourceGroup) {
    if (!$DeleteArmDeployments -or !$ResourceGroup) {
        return
    }
    $toDelete = @()
    try {
        $toDelete = @(Get-AzResourceGroupDeployment -ResourceGroupName $ResourceGroup.ResourceGroupName `
            | Where-Object { $_ -and ($_.Outputs?.Count -or $_.Parameters?.ContainsKey('testApplicationSecret')) })
    }
    catch {}
    if (!$toDelete -or !$toDelete.Count) {
        return
    }
    Write-Host "Deleting $($toDelete.Count) ARM deployments for group $($ResourceGroup.ResourceGroupName) as they may contain output secrets. Deployed resources will not be affected."
    $null = $toDelete | Remove-AzResourceGroupDeployment
}

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

eng/scripts/test-sub-cleanup.ps1:642

• DeleteSubscriptionDeployments is executed unconditionally after DeleteOrUpdateResourceGroups (when not in -DryRun). This means running the script just to tag/delete resource groups will also remove all subscription-scoped ARM deployment history, even if -DeleteArmDeployments wasn’t specified. Gate this call behind the -DeleteArmDeployments switch (or introduce a separate explicit switch) so deployment scrubbing is opt-in and matches the parameter surface.

try {
    DeleteOrUpdateResourceGroups
    if (-not $DryRun) {
        DeleteSubscriptionDeployments
    }

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

eng/scripts/test-sub-cleanup.ps1:30

• ValidatePattern for $TenantId only allows lowercase hex ([0-9a-f]), which can reject valid uppercase GUID inputs. Consider using [0-9a-fA-F] (or (?i) prefix) so the script accepts standard GUID formatting from Azure portal/CLI outputs.

    [Parameter(ParameterSetName = 'Provisioner', Mandatory = $true)]
    [Parameter(ParameterSetName = 'Interactive')]
    [ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
    [string] $TenantId,

eng/scripts/test-sub-cleanup.ps1:35

• ValidatePattern for $SubscriptionId only allows lowercase hex ([0-9a-f]), which can reject valid uppercase GUID inputs. Make the GUID regex case-insensitive or allow A-F so callers don't hit unexpected parameter validation failures.

    [Parameter(ParameterSetName = 'Provisioner', Mandatory = $true)]
    [Parameter(ParameterSetName = 'Interactive')]
    [ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
    [string] $SubscriptionId,

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

eng/scripts/test-sub-cleanup.ps1:30

• ValidatePattern for $TenantId only matches lowercase GUIDs. This will reject valid uppercase GUID inputs copied from the portal/CLI. Update the regex to be case-insensitive so the interactive parameter set is easier to use.

    [Parameter(ParameterSetName = 'Provisioner', Mandatory = $true)]
    [Parameter(ParameterSetName = 'Interactive')]
    [ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
    [string] $TenantId,

eng/scripts/test-sub-cleanup.ps1:35

• ValidatePattern for $SubscriptionId only matches lowercase GUIDs. Valid uppercase subscription IDs will fail validation. Make the GUID regex case-insensitive to avoid false failures.

    [Parameter(ParameterSetName = 'Provisioner', Mandatory = $true)]
    [Parameter(ParameterSetName = 'Interactive')]
    [ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
    [string] $SubscriptionId,