Skip to content

fleet(secrets-broker): SECRETS_BROKER_TTL_S=0 makes the vault-refresh loop spin — ~110 full bws refreshes/sec, no startup validation #5860

Open

Description

@dezren39

fleet(secrets-broker): SECRETS_BROKER_TTL_S=0 makes the vault-refresh loop spin — ~110 full bws refreshes per second, no startup validation

Evidence

pkgs/fleet/crates/fleet-queue/src/secrets_broker.rs:204-207 parses the TTL with no validation:

let ttl_s: u64 = std::env::var("SECRETS_BROKER_TTL_S")
    .ok()
    .and_then(|v| v.parse().ok())
    .unwrap_or(900);

and the background refresh loop at secrets_broker.rs:219-228 sleeps exactly that long between full vault refreshes:

loop {
    match refresh(&state) { ... }
    std::thread::sleep(Duration::from_secs(ttl_s));
}

Each refresh() is a full vault fetch: one bws secret list plus one bws secret get per secret (8-way pool, do_refresh at secrets_broker.rs:111-185).

Verified this run on the laptop with the real standalone secrets-broker binary and a stub bws on PATH that counts invocations (probe ~/git/fleet/probe-secrets-ttl-b, stub returns one secret):

  • SECRETS_BROKER_TTL_S=0: 882 bws calls in 4 seconds (~441 list+get refresh cycles, ≈110 full vault refreshes/sec, sustained — the sleep is zero-length).
  • Controls in a sibling probe (~/git/fleet/probe-secrets-ttl): TTL_S=900 and TTL_S=abc each made exactly 1 refresh (2 calls) in 3s.

Two defects:

  1. Zero (and by extension any pathologically small) TTL is accepted. A single typo'd env value (SECRETS_BROKER_TTL_S=0 in a unit file / Nix module) turns a resident daemon whose whole purpose is to avoid hammering Bitwarden (module doc, secrets_broker.rs:3-7: it replaces per-call bws fetches that cost 10–40s each) into a process that hammers bws continuously, burning CPU and vault API quota for as long as it runs.
  2. An unparseable TTL silently becomes 900. TTL_S=abc (or a negative value, which fails the u64 parse) loads with no warning, so an operator who set e.g. TTL_S=60s gets 900s staleness and no signal why.

Why it matters

The secrets broker sits on the fleet's credential path (repo-queue and every fleet secrets get consumer). A busy-looping broker degrades the laptop it shares with all other fleet work and can trip Bitwarden rate limits — taking down secret access fleet-wide — from one bad env value that startup never questioned. Sibling numeric env/config values in this codebase are startup-validated (e.g. atlas-search validate_limits, code#5630); this one is not.

Acceptance criteria

  • SECRETS_BROKER_TTL_S=0 (and any value below a stated sane minimum) is refused at startup with a clear error naming the variable and value, OR clamped to the minimum with a loud log line — pick one behaviour and pin it with a test.
  • An unparseable SECRETS_BROKER_TTL_S produces a visible warning (log or stderr) naming the value and the 900s default being used, instead of silently defaulting.
  • A test pins the TTL parsing/validation (extract a pure helper if needed): unset → 900, valid value → that value, zero/below-minimum → the chosen refusal/clamp, garbage → default + warning path.
  • Re-probe with the stub-bws harness: at the minimum accepted TTL, refresh cycles in a fixed window match the configured rate (no spin).

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    task/triageQueue packet kind: verify, reproduce, and route this item
    source/creatorFiled by the new-issue-creator job
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    app/fleetpkgs/fleet multicall binary + platform crates
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
  3. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

    • Agent: new-issue-creator job (lane B, new-issue-creator-code-b)
    • Session / ID: cron new-issue-creator-code-b run 2026-11-04 19:50 CST
    • Found via: filed this run
    • Task: Filed after verifying on the laptop: real secrets-broker binary + stub bws — SECRETS_BROKER_TTL_S=0 produced 882 bws calls in 4s (~110 full refreshes/sec); TTL parse at secrets_broker.rs:204-207 has no validation and silently defaults unparseable values to 900.
    • Expectation: effort s — triage packet enqueued to repo-health-code-pilot; done = TTL validated at startup per the acceptance criteria.
    • Links: queue packet hidden_files/packet-5860-triage.json (goal workspace); probe ~/git/fleet/probe-secrets-ttl-b (laptop)
  4. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  5. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  6. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed (enqueue duplicate flag)

  7. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · blocked

  8. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  9. added
    queue/blockedQueue or user escalation state
    status/blockedQueue or user escalation state
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/fleetpkgs/fleet multicall binary + platform crateskind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)queue/blockedQueue or user escalation statesource/creatorFiled by the new-issue-creator jobstatus/blockedQueue or user escalation statetask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions