Repository navigation
ci(github-backup): the infra/github-backup regression suite (10 files, ~250 checks) is run by no workflow #5937
Description
Finding
The infra/github-backup regression suite — 10 test files under infra/github-backup/tests/, ~250 checks over the org backup estate (clone/mirror, reconcile retention, queue drain, worker deploy/rollback) — is run by no workflow in the repo:
grep -rn "github-backup" .github/workflows/*.ymlreturns nothing (verified on laptop primary @9ddf6a8216, 2026-11-05). There is nogithub-backupworkflow at all; the systemd units underinfra/github-backup/systemd/run the backup jobs, not the tests.- The suite's own README entry (
infra/github-backup/README.md:68) describestests/only as "regression tests (plain python3, no deps)" — no documented CI runner exists.
Same gap shape as closed #1427 (cold-store-reaper tests executed only when a human remembered — fixed by adding a workflow) and #5582 / #5935, in the backup subsystem.
Verified state of the suite
Run on the laptop @ 9ddf6a821658 (HEAD == origin/main), each file as a plain script — all 10 pass:
| file | result |
|---|---|
| test_clone_all_partial.py | pass |
| test_clone_all_references.py | 4/4 |
| test_deploy_dry_run.py | 11/11 |
| test_deploy_rollback_version.py | 16/16 |
| test_deploy_unknown_args.py | 53/53 |
| test_drain_queue.py | 82/82 |
| test_fetch_mirrors.py | OK |
| test_inventory_owner.py | 18/18 |
| test_reconcile_clone_pass.py | 9/9 |
| test_reconcile_retention.py | 9/9 |
These tests guard safety-critical behaviour that has already bitten once: the deploy --dry-run --deploy-version combination that performed a real 100% deployment (#3474), rollback verify-before-deploy (#3475), retention of a failed owner's mirrors in plan/MANIFEST, and the drain-queue contract. A regression in any of them currently surfaces only when someone happens to run the files by hand.
Wiring caveat (verified)
A naive python3 -m unittest discover -s infra/github-backup/tests step does not work as-is: the three test_deploy_*.py modules execute their checks at import time and end in sys.exit(...) at module level (e.g. test_deploy_dry_run.py:112), so discovery reports FAILED (errors=3) with SystemExit: 0 even though every check passed. The other seven files are a mix of unittest classes and the same script style. The wiring must either run each file as a script (python3 tests/test_*.py, the documented plain-python3 mode) or first move the script-style checks behind a __main__ guard / into TestCase classes.
Acceptance criteria
- Some workflow runs all 10 files in
infra/github-backup/tests/and fails when any file exits non-zero (a dedicated workflow or an added job/step in an existing scripts/infra workflow; dispatch-triggered is consistent with the house pattern since5a94833ea3). - The wiring copes with the script-style modules (runs them as scripts, or the modules are restructured so
unittest discoveris clean) — a green run that silently skipped the three deploy suites does not count. - Full suite passes under the new wiring on the workflow's Python.
- No behaviour change to
infra/github-backup/bin/*,worker/*, or the systemd units.
Evidence
- File/line and grep evidence above, read on laptop primary
~/git/code@9ddf6a821658c8700fb78c06b024e2a1b024f754. - Suite runs on laptop, 2026-11-05: all 10 files exit 0 as scripts (results table above);
unittest discoverover the directory reportsFAILED (errors=3)solely from the three import-timesys.exitdeploy modules. - Related: ci(cold-store-reaper): no workflow runs the package's test suites — 2,700+ lines of tests execute only when a human remembers #1427 (closed precedent, same shape), github-backup(deploy): --dry-run is ignored on the --deploy-version path — a rollback preview creates a real 100% deployment #3474 / github-backup(deploy): the version advertised for rollback is the pre-secrets version, and --deploy-version never provisions secrets #3475 (incidents the deploy tests pin), github-backup: drain-wrapper Bitwarden fallback (PRs #3222/#3226) has no test — the bws-env branch runs only where the primary path already failed #5222 (one specific untested drain-wrapper branch — distinct from this whole-suite wiring gap).
Activity
developing-today commentedon Nov 5, 2026
CODE-1610
dezren39 commentedon Nov 5, 2026
🤖 Agent note · filed
new-issue-creator-code-b)new-issue-creator-code-b, run 2026-11-04 21:20 CSTinfra/github-backupregression suite (10 test files, ~250 checks) is run by no workflow; verified on laptop primary @ 9ddf6a8 by running all 10 files as plain scripts (all exit 0) and by a workflow grep returning zero references, plus the verifiedunittest discovercaveat (3 deploy modules exit at import).packet-5937-triage.jsonenqueued to repo-health-code-pilot this run.dezren39 commentedon Nov 5, 2026
🤖 Agent note · picked up
Merge pull request #5938 from developing-today/queue/9835-5937-github…
dezren39 commentedon Nov 5, 2026
🤖 Agent note · parked
github-backup testsworkflow was dispatched on main (run 41963838314) but stayed queued >5 minutes (runner congestion), so task 9835 is parked rather than held. The suite itself passed all 10 files as scripts on the laptop pre-merge, and the workflow step's failure propagation was negative-tested.dezren39 commentedon Nov 5, 2026
Agent note — resumed, CI still pending, re-parked (task 9835)
github-backup testsworkflow run 41963838314 (headSha a328819, workflow_dispatch, created 2026-11-05T03:48:07Z) is STILLqueuedwith no conclusion as of 2026-11-05T04:19Z — runner congestion, unchanged after a further 60s poll. It is the only run of that workflow. The suite itself was verified green on the laptop pre-merge by the previous worker, including a negative failure-propagation test.