[ci-scan-feedback] Guard KBE signatures against shared multi-test Helix console-log catch-alls

[github-actions]

Triggering signals

• [ci-scan] Test failure: profiler/gc/nongcheap/nongcheap.cmd fails with missing 'PROFILER TEST PASSES' #131449 / tracker [ci-scan-feedback] KPI Tracker #128742 — noahfalk [MEMBER]: the profiler KBE matched the shared multi-test Helix console log, pairing a per-run Running test: banner with a generic profiler-harness error line, so every profiler test failure in that workitem bucketed onto the one KBE. He manually corrected the KBE and asked that generation avoid matching shared-console error text, or capture a single expression that is both error-indicating and test-specific. ([ci-scan] Test failure: profiler/gc/nongcheap/nongcheap.cmd fails with missing 'PROFILER TEST PASSES' #131449, [ci-scan-feedback] KPI Tracker #128742)
• Rubric finding: create-kbe.instructions.md's existing PASS/SKIP hard gate (§7) catches signatures that match [PASS]/[SKIP] lines, but does not catch a signature that matches sibling tests' failures in a shared console log — a distinct catch-all failure mode.

Proposed edits

• .github/workflows/shared/create-kbe.instructions.md (§7, after the PASS/SKIP negative-output gate) — add a "Shared multi-test Helix console log (catch-all gate)" rule: when the log is a Helix per-workitem console log running many tests, require the most-specific element to be unique to the failing test (not the workitem), with three remediations (extend the array with a failing-path-only line, set ExcludeConsoleLog: true + a leg/xunit-level signature, or skip with shared-log catch-all risk, needs human review). Rationale: directly addresses noahfalk's [ci-scan] Test failure: profiler/gc/nongcheap/nongcheap.cmd fails with missing 'PROFILER TEST PASSES' #131449 over-matching.
• .github/workflows/shared/create-kbe.instructions.md (Bad vs good signatures table) — add a Bad/Good row using the concrete [ci-scan] Test failure: profiler/gc/nongcheap/nongcheap.cmd fails with missing 'PROFILER TEST PASSES' #131449 profiler pattern (banner + harness message → catch-all) so the rule is illustrated by example.

Expected behavior change

The next scanner run will no longer file a KBE whose signature, though it greps positive and avoids PASS/SKIP lines, is scraped from a shared Helix console log and pairs a per-run banner with a generic error message. Such signatures either gain a failing-test-specific anchor, switch to a leg-level signature via ExcludeConsoleLog: true, or are skipped for human review — preventing the catch-all buckets that combined unrelated profiler failures onto a single KBE.

Note

This pull request was generated by an AI agent (GitHub Copilot).

Generated by CI Outer-Loop Failure Scanner — Feedback · opus48 · 443.4 AIC · ⌖ 25.3 AIC · ⊞ 20.6K · ◷

---

Note

This was originally intended as a pull request, but the git push operation failed.

Original error: The process '/usr/bin/git' failed with exit code 1

Workflow Run: View run details and download bundle artifact

The bundle file is available in the agent artifact in the workflow run linked above.

To create a pull request with the changes:

# Download the artifact from the workflow run
gh run download 32574800673 -n agent -D /tmp/agent-32574800673

# Fetch the bundle into a temporary ref, then update the local branch
git fetch /tmp/agent-32574800673/aw-ci-scan-feedback-console-log-catchall-20260822.bundle refs/heads/ci-scan-feedback/console-log-catchall-20260822:refs/bundles/create-pr-ci-scan-feedback-console-log-catchall-20260822-93341558b4632209-22d6adf7
git update-ref refs/heads/ci-scan-feedback/console-log-catchall-20260822-93341558b4632209 refs/bundles/create-pr-ci-scan-feedback-console-log-catchall-20260822-93341558b4632209-22d6adf7
git checkout ci-scan-feedback/console-log-catchall-20260822-93341558b4632209
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d refs/bundles/create-pr-ci-scan-feedback-console-log-catchall-20260822-93341558b4632209-22d6adf7

# Push the branch to origin
git push origin ci-scan-feedback/console-log-catchall-20260822-93341558b4632209

# Create the pull request
gh pr create --title '[ci-scan-feedback] Guard KBE signatures against shared multi-test Helix console-log catch-alls' --base main --head ci-scan-feedback/console-log-catchall-20260822-93341558b4632209 --repo dotnet/runtime