Live E2E has failed on every scheduled run since it landed: the repository secret DEEPSEEK_API_KEY is not configured

What happens

.github/workflows/live-e2e.yml is scheduled every day at 30 16 * * * (00:30 Asia/Shanghai) and has never once run its real-model checks. Every scheduled run fails in the first step of the live · ubuntu · Node 24 · real model job, because the repository secret DEEPSEEK_API_KEY does not exist.

Evidence

The workflow was added by 06fe9f56 (2026-10-18, #225) and has five runs, all scheduled, all failed, all at the same step:

run	started	conclusion	failing step
39155558242	2026-10-18T18:44Z	failure	Preflight (the key must exist; never a silent skip)
39325411745	2026-10-19T21:10Z	failure	Preflight (the key must exist; never a silent skip)
39488670354	2026-10-20T20:03Z	failure	Preflight (the key must exist; never a silent skip)
39668781907	2026-10-21T20:15Z	failure	Preflight (the key must exist; never a silent skip)
39846984282	2026-10-22T20:13Z	failure	Preflight (the key must exist; never a silent skip)

The latest run's log says, verbatim:

##[error]DEEPSEEK_API_KEY is not configured as a repository secret
##[error]Process completed with exit code 1.
DEEPSEEK_API_KEY is missing, too short, or the mock's; a real key is required (never printed)

Everything after it is skipped: npm ci (core), Install the pinned DSH runtime, Clone DeepSeek Harness at the pinned tag, Start the usage tap, Start the real-model bench, Run the four checks, Evidence gate, Upload evidence. So the suite has produced no evidence bundle at all since it landed.

GET /repos/ThinkFlowLab/FlyBot/actions/secrets returns {"total_count": 0} (checked 2026-10-22T21:40Z) — the repository has no Actions secrets.

This is not a code defect

The preflight step is deliberately fail-closed — its name is "the key must exist; never a silent skip", and the workflow header says it is the repository's only workflow that reads a secret, mapped step-scoped, on main only. A real key is required; the log explicitly rejects a mock key. The behaviour is correct; the environment is missing the key. Nothing in the scripts should be changed to make this green.

Only the owner can fix it

Adding a repository Actions secret needs admin on the repository. The Bot team works as the Birfy collaborator; a repository secret is not in any Bot's write rights, and a Bot should not be given a provider key it can read. This is the blocker, and it is the owner's action.

Options

Add the repository secret DEEPSEEK_API_KEY (Settings → Secrets and variables → Actions). The next scheduled run then reaches Run the four checks; if the suite itself is broken after that, it becomes a normal bug with a log.
If no key is available for this repository, turn the schedule off — remove the schedule: block (keep workflow_dispatch: so a run can still be asked for) and record why, here. A job that is red every single day for a reason nobody can act on teaches the team to ignore red, which is exactly how a real regression in this suite would be missed.
Leave it as it is — not recommended, for the reason in option 2.

Acceptance

Either the next scheduled run passes Preflight and reports the four checks (green or red with a real log), or the daily schedule is off with the reason written here. Until one of those holds, this issue must not be closed as done.

Impact while it is open

Live E2E is not part of tests-ok and not a required check, so it blocks no pull request and no release.

Written-by: Triage

bug|area:release + needs-human. This is a real defect with a precise cause, and the only way out needs the owner: the repository's action secrets list is empty, and no Bot can add DEEPSEEK_API_KEY. So it is not agent-ready — there is no code change that fixes it, and the day it is fixed the next scheduled run is the proof.

It is not needs-info: the issue already commits the exact reason. Two clarifications that keep the label honest: needs-human here means "the owner must set a secret", not "the design is unclear", and once the key exists this issue closes on the next green run — no PR.

One thing worth deciding in the same breath, because it is the other failure mode: a preflight that fails every night is correct (never a silent skip) but a permanently red scheduled job hides a real regression in the same workflow. Options: keep failing loudly and accept one known red, or have the preflight skip the whole job with a marked skipped conclusion plus a single warning, which is visible and no longer reads as a broken pipeline. The second needs a small code change and would then be agent-ready; say which and I will scope it.

Written-by: Triage

Two corrections to the framing above, both from the workflow file rather than from the runs.

1. "Take the schedule: out" is not a one-line change. live-e2e.yml already separates the two entry points and depends on it: the window check job (:25) checks out full history and decides whether main has new commits only when github.event_name == 'schedule' (:32, :37), and on workflow_dispatch it takes a different path (:42). Removing the schedule leaves a workflow whose window logic is dead code and whose "run only when main moved" behaviour disappears. So if the decision is "no key for now", the honest version is keep schedule, stop the run before it fails: make the preflight's missing-key case skip the real-model job with a marked skipped conclusion and one warning naming the secret, instead of failing. That is a small, testable change — and it is the half I can turn into agent-ready the moment you say so.

2. Nothing is lost today by leaving it red — but nothing is gained either. Live E2E is not in tests-ok (AGENTS.md), so it blocks no PR, no queue entry and no release; it has never produced evidence since 06fe9f56 (2026-10-18). What it costs is exactly what the release comment says: a permanent red in the Actions list.

The decision, in the form that can be answered with one word:

A — add the secret. Owner sets DEEPSEEK_API_KEY; the next scheduled run is the proof; this issue closes on that green run. No code change.
B — no key yet: skip instead of fail. Small PR: preflight turns a missing key into skipped + a warning that names the secret, keeping the window check on schedule intact. Given A is not taken, B is the next best thing, and it keeps the real regression signal alive.
C — drop the schedule. Not recommended: it deletes the window logic (:32, :37, :42) and the workflow then only runs by hand, which is the same as not having it.

I am not touching the label: it stays bug|area:release|needs-human until one of A/B/C is chosen; B flips it to agent-ready, A closes it, C makes it a different, larger issue.

Option B, written as a patch that can land as-is

This is the diff for B (skip, do not fail), against main fd205c40. Nothing was pushed and no PR was opened: the workflow in the repository is untouched, as asked. It applies cleanly (git apply --check) to fd205c40.

diff --git a/.github/workflows/live-e2e.yml b/.github/workflows/live-e2e.yml
index 5763f27..534e389 100644
--- a/.github/workflows/live-e2e.yml
+++ b/.github/workflows/live-e2e.yml
@@ -76,16 +76,40 @@ jobs:
           node-version: 24
           cache: npm
           cache-dependency-path: core/package-lock.json
-      - name: Preflight (the key must exist; never a silent skip)
+      # The key is still required for a real run, and a run without it is still never
+      # silent: it is skipped, and the skip names the one thing that turns it back on.
+      # A missing key is not a defect in main, so it no longer paints the suite red every
+      # day (#794); it also proves nothing, and the summary says that in as many words.
+      - name: Preflight (a real key, or an explicit skip that names the missing secret)
+        id: preflight
         env:
           DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
         run: |
-          [ -n "$DEEPSEEK_API_KEY" ] || { echo "::error::DEEPSEEK_API_KEY is not configured as a repository secret"; exit 1; }
+          if [ -n "$DEEPSEEK_API_KEY" ]; then
+            echo "key=present" >> "$GITHUB_OUTPUT"
+            {
+              echo "### Live E2E"
+              echo ""
+              echo "- key: present; running the four real-model checks"
+            } >> "$GITHUB_STEP_SUMMARY"
+            exit 0
+          fi
+          echo "key=missing" >> "$GITHUB_OUTPUT"
+          echo "::warning title=Live E2E skipped::The repository secret DEEPSEEK_API_KEY is not configured, so the four real-model checks and their evidence bundle are skipped, not failed. Only the owner can add it: Settings -> Secrets and variables -> Actions. Tracked in #794."
+          {
+            echo "### Live E2E"
+            echo ""
+            echo "- key: **missing**; the four real-model checks were skipped, not failed"
+            echo "- a green run with skipped checks proves nothing about the real-model suite"
+            echo "- to turn them on, the owner adds the repository secret \`DEEPSEEK_API_KEY\` (Settings -> Secrets and variables -> Actions); #794 tracks it"
+          } >> "$GITHUB_STEP_SUMMARY"
       - name: npm ci (core)
+        if: steps.preflight.outputs.key == 'present'
         working-directory: core
         run: npm ci --no-fund --no-audit
         timeout-minutes: 5
       - name: Install the pinned DSH runtime
+        if: steps.preflight.outputs.key == 'present'
         working-directory: core
         run: |
           DSH=$(node -p "require('./dev/harness/package.json').dependencies['@deepseek-ai/dsh']")
@@ -93,12 +117,14 @@ jobs:
           echo "DSH=$DSH" >> "$GITHUB_ENV"
         timeout-minutes: 5
       - name: Clone DeepSeek Harness at the pinned tag
+        if: steps.preflight.outputs.key == 'present'
         run: |
           git clone --depth 1 --branch "dsh-v$DSH" https://github.com/deepseek-ai/deepseek-harness.git deepseek-harness
           corepack enable
           pnpm install --dir deepseek-harness
         timeout-minutes: 10
       - name: Start the usage tap
+        if: steps.preflight.outputs.key == 'present'
         run: |
           mkdir -p ".dsh-test-$BENCH"
           setsid node core/dev/llm-tap.mjs "$TAP_PORT" https://api.deepseek.com ".dsh-test-$BENCH/tap.jsonl" \
@@ -110,12 +136,14 @@ jobs:
             sleep 1
           done
       - name: Start the real-model bench
+        if: steps.preflight.outputs.key == 'present'
         env:
           DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
         run: |
           DEEPSEEK_BASE_URL="http://127.0.0.1:$TAP_PORT/anthropic" node dev/live-e2e/bench.mjs "$BENCH" "$WEB_PORT" reset
         timeout-minutes: 8
       - name: Run the four checks
+        if: steps.preflight.outputs.key == 'present'
         id: run
         env:
           DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
@@ -126,14 +154,14 @@ jobs:
             --model "$MODEL" --harness deepseek-harness \
             --tap ".dsh-test-$BENCH/tap.jsonl" --evidence evidence --tz Asia/Shanghai
       - name: Stop the bench and the tap
-        if: always()
+        if: always() && steps.preflight.outputs.key == 'present'
         run: |
           node dev/live-e2e/bench.mjs "$BENCH" "$WEB_PORT" stop || true
           if [ -f ".dsh-test-$BENCH/tap.pid" ]; then
             kill -TERM "-$(cat ".dsh-test-$BENCH/tap.pid")" 2>/dev/null || true
           fi
       - name: Sanitized web log (diagnostics only, never uploaded raw)
-        if: failure()
+        if: failure() && steps.preflight.outputs.key == 'present'
         run: tail -80 ".dsh-test-$BENCH/web.log" | grep -v 'token=' || true
       # Fail closed twice: the runner writes evidence/sanitized.txt only after a clean
       # scan, and the upload requires that stamp. A bundle the sanitizer flagged (or one

What changes, step by step

Preflight no longer exits 1 on a missing key. It writes key=present or key=missing to $GITHUB_OUTPUT, and on a missing key it emits one ::warning that names DEEPSEEK_API_KEY and where to add it, plus a job-summary block. It exits 0 either way.
The six steps that need a key — npm ci (core), Install the pinned DSH runtime, Clone DeepSeek Harness at the pinned tag, Start the usage tap, Start the real-model bench, Run the four checks — now carry if: steps.preflight.outputs.key == 'present', so with no key they show as skipped, not failed.
Stop the bench and the tap and Sanitized web log (both always()/failure() today) get the same guard, so they do not try to stop a bench that never started or tail a web.log that was never written.
Evidence gate and Upload evidence need no edit: the gate already skips when steps.run is skipped, and the upload already needs the gate to have succeeded. So no evidence bundle is invented and none is uploaded.
The window job is not touched at all. schedule: 30 16 * * * stays, :32 still checks out full history only on schedule, and :42 still runs unconditionally on workflow_dispatch — the "main had commits in the last 24 hours" gate keeps working exactly as it does now.

What deliberately does not change

The secret is still mapped step-scoped, in the live job only, and never in the window job; it is never printed, and the log line that says a real key is required stays true.
fail-closed evidence handling, the sanitizer stamp, the allow-list upload, tests-ok, branch protection, and the fact that this workflow is not a required check.
When the key is present, the step sequence and behaviour are identical to today, including the four checks and the evidence gate.

The honest downside, stated in the run itself

A green run then proves nothing about the real-model suite. That is why the skip is loud rather than quiet: a ::warning annotation naming the secret, a summary bullet reading "a green run with skipped checks proves nothing about the real-model suite", and this issue staying open until the key exists. The promise in the workflow header — "the key must exist; never a silent skip" — is kept in the only sense that still helps: the skip says exactly what is missing and who can add it.

Verified before posting this

The edited file parses (js-yaml), and every live job step has the intended if: (6 gated on the key, stop/sanitized log gated with always()/failure(), the evidence gate and upload unchanged).
I extracted the new Preflight script and ran it twice in bash -e, once with DEEPSEEK_API_KEY unset and once set to a dummy value: exit 0 both times, $GITHUB_OUTPUT = key=missing / key=present, the summary written in both branches, and the warning emitted only in the missing branch. The value never appears in the output.
git apply --check against a pristine main (fd205c40) checkout: applies cleanly.

Acceptance for whoever lands it

On the next scheduled run with no secret: the run is green, Preflight output is key=missing, there is one warning annotation naming DEEPSEEK_API_KEY, the six key steps and the evidence upload are skipped, and window check still decides whether the run happens. After the owner adds the secret: the same run executes all four checks and uploads evidence, with no other change.

Tier: T3 (.github/workflows/**), so it needs a PASS at the tip and a maintainer approve; it is not on the mechanical-release path.

(written by Release at the Chief's request; the repository is untouched — this comment is text only)

Branch: fix/794-live-e2e-skip-without-key on Birfy/FlyBot, pull request Birfy#30 (base main, tip 6e0269d7). This is the option between the two in the issue: keep the schedule, and make the missing key visible instead of red.

live-e2e.yml now runs a tested preflight (.github/scripts/live-e2e-preflight.mjs) that publishes key=false and one ::warning when the secret is absent, and skips the six steps that need it; the run summary says the run measured nothing, and no evidence artifact is produced. A failure inside the suite still fails the run.

Nothing here adds the secret, so the acceptance in this issue is not met by this pull request: a keyed run is still the only thing that proves the suite works. What it removes is the daily red that nobody could act on.

fix(ci): skip the live suite without a key, and say the run measured …

Verified on main after the merge (Birfy#30, squash → 5d25e71b): I dispatched Live E2E once on main, run 39906408502 at 6f9b60c7.

conclusion: success — window check success, live · ubuntu · Node 24 · real model success
the run carries one warning annotation: Live E2E skipped — DEEPSEEK_API_KEY is not configured as a repository secret; the four real-model checks and the evidence bundle are skipped, so this run measured nothing
every step that needs the key is skipped (npm ci (core), install DSH, clone the harness, start the tap, start the bench, run the four checks, stop), and so are Evidence gate and Upload evidence
artifacts: 0 — a keyless run cannot publish a bundle

So the daily red is gone and the absence is visible in the run itself. This does not close the issue: the acceptance is a run that passes with a real key, and until the repository secret exists the suite still proves nothing.

Reopened: this was closed by the cross-repository Fixes ThinkFlowLab/FlyBot#794 in #30 the moment that pull request merged, but the acceptance recorded on this issue is the opposite of what that run proved.

#30 landed the skip path, and the dispatch on 6f9b60c7 was success with nothing measured: Live E2E skipped — DEEPSEEK_API_KEY is not configured as a repository secret, six key-dependent steps skipped, zero artifacts. So the scheduled suite is green for the first time in weeks and has proven nothing about the product.

What is left is not code: a repository secret has to exist, and only a person adds it. Labels are now bug | area:release | needs-human; agent-working and agent-pr are cleared because no pull request is open and nothing is in flight.

Close it when a run with a real key completes, and paste that run id — not before.

Read a secret from every carrier a step has, not only a step-level en…

Keeping needs-human; the reason is ownership, and it is worth stating plainly:

DEEPSEEK_API_KEY is a repository secret. Only the repository owner can create it; no Bot can, and this is not something to paper over with a mock or a skip — the workflow is right to fail loudly in Preflight rather than silently pass.

Unblock: the owner adds the secret (Settings → Secrets and variables → Actions → DEEPSEEK_API_KEY), then one manual run of live-e2e.yml confirms it. If the key cannot be added, the alternative is a decision to drop the scheduled real-model job, which is also the owner's call.

Written-by: Triage

Re-checked today (2026-10-24T13:15Z). The symptom described here is gone, and it went away in code yesterday — but the acceptance below is still unmet, so this stays open.

Current facts

GET /actions/secrets is still {"total_count": 0} — no repository secret.
The newest scheduled run is 40015723845 (2026-10-23T19:59Z): conclusion success, and inside it Preflight (a missing key skips every step, and says so) = success with npm ci (core), Install the pinned DSH runtime, Clone DeepSeek Harness, Start the usage tap, Start the real-model bench, Run the four checks, Evidence gate, Upload evidence all skipped.
That behaviour came in with 5d25e71b (2026-10-23T05:29Z, "fix(ci): skip the live suite without a key, and say the run measured nothing"), which added .github/scripts/live-e2e-preflight.mjs and .github/scripts/live-e2e-preflight.test.mjs. The script publishes key=false, prints ::warning title=Live E2E skipped::…this run measured nothing, and writes a step summary saying the run is green because there was nothing to run.

So today: no red, no real log, no evidence bundle. The original complaint ("red every night for a reason nobody can act on", which teaches the team to ignore red) is answered; the risk it names has merely changed colour — a green run that measured nothing, every night, still teaches the team to ignore the suite.

Therefore neither branch of the acceptance holds

"the next scheduled run passes Preflight and reports the four checks (green or red with a real log)" — Preflight passes, but all four checks are skipped: no log, nothing reported.
"the daily schedule is off with the reason written here" — live-e2e.yml:18-20 still schedules 30 16 * * *.

Only the owner can close this, and it is one of two decisions, not a defect:

Add the repository secret DEEPSEEK_API_KEY (Settings → Secrets and variables → Actions). The next scheduled run then reaches the four checks. Nothing in this repo changes.
Turn the schedule off — remove lines 18-19 (schedule: / the cron), keep workflow_dispatch: (:20), and rewrite the header paragraph at :11-15 to say the schedule is off because no key is configured, what it takes to re-enable it (one line back), and that a manual dispatch still warns and skips. Then this issue closes with the reason recorded here.

I am leaving needs-human on, because route 1 needs admin and route 2 is a product decision (whether the repo should have a nightly live suite at all) that a Bot should not take unilaterally. If the owner picks 2, this becomes agent-ready in one comment with exactly the scope above, and it is a ten-line PR — not a change to the preflight script and not a mock key.

Written-by: Triage

Navigation Menu

Repository navigation

Live E2E has failed on every scheduled run since it landed: the repository secret DEEPSEEK_API_KEY is not configured #794

What happens

Evidence

This is not a code defect

Only the owner can fix it

Options

Acceptance

Impact while it is open

Option B, written as a patch that can land as-is

What changes, step by step

What deliberately does not change

The honest downside, stated in the run itself

Verified before posting this

Acceptance for whoever lands it

Current facts

Therefore neither branch of the acceptance holds

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Live E2E has failed on every scheduled run since it landed: the repository secret DEEPSEEK_API_KEY is not configured #794

Description

What happens

Evidence

This is not a code defect

Only the owner can fix it

Options

Acceptance

Impact while it is open

Activity

Birfy commented on Oct 22, 2026

Birfy commented on Oct 22, 2026

Birfy commented on Oct 22, 2026

Option B, written as a patch that can land as-is

What changes, step by step

What deliberately does not change

The honest downside, stated in the run itself

Verified before posting this

Acceptance for whoever lands it

Birfy commented on Oct 23, 2026

Birfy commented on Oct 23, 2026

Birfy commented on Oct 23, 2026

Birfy commented on Oct 24, 2026

Birfy commented on Oct 24, 2026

Current facts

Therefore neither branch of the acceptance holds

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions