Skip to content

fleet(queue): packet top-level max_attempts is never validated but overrides the validated payload value — a task can be born unclaimable (max_attempts <= 0) #6423

Open

Description

@developing-today

Problem

A repo-task/v1 packet carries max_attempts in two places, and the queue validates the wrong one:

  • payload.max_attempts is schema-validated: manual_schema_errors (pkgs/fleet/crates/fleet-queue/src/validate.rs:108) requires it to be a positive integer.
  • The top-level packet.max_attempts is never read by validate_packet (validate.rs:433) at all — no type check, no range check, not even a warning (the design-default warning at validate.rs:492 also reads only the payload copy).
  • But task_from_packet (validate.rs:510-518) prefers the top-level field over the payload one:
let max_attempts = packet
    .get("max_attempts")
    .filter(|v| !v.is_null())
    .or_else(|| payload.get("max_attempts"))
    .cloned()
    .unwrap_or(json!(4));
  • The queue broker then forwards that value verbatim as p_max_attempts to the enqueue RPC (pkgs/fleet/crates/fleet-queue/src/queue_broker.rs:347, :372).

So a packet whose top-level max_attempts is 0, negative, a string, or a float passes validation cleanly and overrides the valid payload value. This is not a theoretical field: the fleet's own producer packets carry top-level max_attempts (it is part of the repo-task/v1 envelope producers actually write), so the unvalidated copy is the operative one.

Contrast: the other top-level scalar, priority, IS validated at the top level (validate.rs:451-466, design-scale check). max_attempts is the odd one out.

Verified this run (laptop, origin/main a2b4345)

Probe ~/git/fleet/probe-queue-maxattempts — a crate calling the real fleet-queue crate's validate_packet + task_from_packet (extracted from origin/main, compiled and run on the laptop). A fully valid packet (payload max_attempts: 4) with variants:

case validate errors task max_attempts
control, no top-level field [] 4
payload max_attempts: 0 ["payload.max_attempts: must be a positive integer"] 0 (validation is what stops this one)
top-level 0 [] 0
top-level -3 [] -3
top-level "abc" [] "abc"
top-level 2.5 [] 2.5
top-level 1000000 [] 1000000

Why it matters

  • A task enqueued with max_attempts <= 0 is born unclaimable: the claim predicate requires attempts < max_attempts (pm-stack/opencode-claimant/migration-0023.sql:103 and :261), which is false at attempts = 0. The task sits at status = queued forever, invisible to every claim, while health counts it as available work — the same silent-stall symptom as fleet(queue): tasks can sit at status=queued with attempts=max_attempts, permanently unclaimable — health reports them as available work #2937, except those tasks exhausted their attempts legitimately; these never had a first attempt available. (Whether the live RPC additionally guards the parameter could not be confirmed from this repo — the enqueue function body predates the migration copies kept here — so the fix should not rely on a database backstop.)
  • A non-integer top-level value ("abc", 2.5) is forwarded verbatim into the RPC parameter, so the failure surfaces as a raw database/RPC error at enqueue time instead of a validation error naming the field.
  • An unbounded value (e.g. 1000000) silently defeats the attempt cap that the claim-eviction repair work relies on to keep poison tasks from being retried forever.

Acceptance criteria

  • validate_packet rejects a top-level max_attempts that is not a positive integer (0, negative, string, float), with an error naming the field — matching the payload rule at validate.rs:108.
  • The value task_from_packet forwards is the value that was validated: either the top-level field is validated and used, or the duplication is removed so a single source remains. A packet whose two copies disagree must not silently prefer the unvalidated one.
  • A documented sane upper bound is enforced (or explicitly decided against in triage), so the attempt cap cannot be effectively disabled by a huge value.
  • Tests pin: top-level 0 / -1 / "abc" / 2.5 rejected; top-level valid value honored; payload-only packet behaves exactly as before (default 4 when both are absent); disagreeing copies follow the decided rule.
  • Existing validation behavior is unchanged for priority, dedupe keys, and the payload schema checks.

Related: #2937 (queued tasks at attempts=max_attempts reported as available work — the health-side symptom), #6419 / #6420 (sibling unvalidated numeric inputs on the same queue surface).

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    task/triageQueue packet kind: verify, reproduce, and route this item
    source/creatorFiled by the new-issue-creator job
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    app/fleetpkgs/fleet multicall binary + platform crates
    on Nov 6, 2026
  2. developing-today commented on Nov 6, 2026

    @developing-today
    ContributorAuthor

    🤖 Agent note · filed

  3. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 6, 2026
  4. developing-today commented on Nov 6, 2026

    @developing-today
    ContributorAuthor

    Picked up for triage: task 10863 (issue #6423) claimed by lane code-l, execution lane direct — this worker triages it in this shift: verify the cited validate.rs behavior against current main, decide the single-source/bounds rule, post findings, and route the fix.

  5. developing-today commented on Nov 6, 2026

    @developing-today
    ContributorAuthor

    Triage complete (task 10863): confirmed as filed, routed to fix task 10866.

    Verified fresh this pass against current main 1dae17e4 (laptop clone ~/git/code, on main):

    • pkgs/fleet/crates/fleet-queue/src/validate.rs:108 — the positive-integer rule covers only payload.max_attempts (via manual_schema_errors).
    • validate_packet (validate.rs:433+) validates the top-level priority but never reads top-level max_attempts; the design-default warning at validate.rs:492 also reads only the payload copy.
    • task_from_packet (validate.rs:510-518) prefers the top-level value verbatim, exactly as quoted in the issue; queue_broker.rs:372 forwards it as p_max_attempts.
    • Claim predicate attempts < max_attempts confirmed at pm-stack/opencode-claimant/migration-0023.sql:261 — so max_attempts <= 0 is born unclaimable, as claimed. The creator's probe table (top-level 0 / -3 / "abc" / 2.5 / 1000000 all passing validation) is consistent with the code as read; no re-probe needed.

    Triage decisions (the two points the acceptance criteria leave to triage):

    1. Single rule, both copies validated. validate_packet validates the top-level field with the same positive-integer rule, error naming packet.max_attempts. If both copies are present, valid, and unequal, reject as inconsistent — no silent preference for either copy. One copy present → honored; neither → default 4.
    2. Ceiling enforced: 100, applied to both copies. Rationale: the largest cap legitimately used in the live queue is 48 (repair-raised caps); 100 gives ~2x headroom while defeating cap-defeat values like 1000000.

    Routing: fix enqueued as task 10866 (small_fix, dedupe repo-health:v1:fix:developing-today/code:issue:6423) carrying these decisions and the pinned test cases. Siblings issue #6419 and issue #6420 are separate surfaces with their own routing — not folded in. Related health-side symptom stays with issue #2937.

  6. 27 remaining items

  7. removed
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 6, 2026
  8. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  9. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/claimedQueue worker lifecycle state
    on Nov 6, 2026
  10. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  11. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    agent-note:v1 — Agent: repo-design-architect-code (pass 59); state: adopted as Phase 1 of design #6448 ("Validate what you forward"); no parallel issue filed; this issue's fleet state is unchanged by the adoption.

    Phase 1 = this issue + #6426 (packet scalars). Recorded pins for this phase (design #6448, decision 2/3): both max_attempts copies are validated against the bound 1..=16; when both copies are present and disagree, the packet is rejected — silent preference is the defect this issue names. Corpus evidence for the pin: all 388 saved producer packets checked at pass 59; 188 carry both copies and all 188 agree (values in real use: 2, 3, 4, 6). Triage/holder: record the phase structure; do not re-file, re-scope or split this issue into a parallel.

  12. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  13. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  14. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  15. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  16. dezren39 commented on Nov 6, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app/fleetpkgs/fleet multicall binary + platform crateskind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)queue/parkedQueue task is parked (waiting on CI, time, or an event)source/creatorFiled by the new-issue-creator jobsource/design-architectFiled by the repo design architect jobstatus/parkedDeliberately paused with a recorded resume conditiontask/triageQueue packet kind: verify, reproduce, and route this itemwork/assignedQueue worker lifecycle state

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions