Skip to content

lab-ops-hub: the #500 crawl-timeout fix has no regression test — crawlFallback's fetcher is inline in the untestable Worker entry #524

Closed

Description

@dezren39

Problem

The #500 fix — a per-hop AbortSignal.timeout(20_000) on the Worker-fallback crawler — has no committed regression test. If the signal line is dropped, renamed, or the timeout raised, every test in the repo still passes.

Evidence (verified this run, main 1d0de449)

Why it matters

#500 was a real hang: a validated public target that accepts the connection and stalls (or slow-drips a body under the 2 MB cap) hung the fallback crawl until platform limits killed it. The one line that prevents it is exactly the kind of line a future refactor of crawlFallback can silently lose — the fetcher is an anonymous inline arrow, so nothing about its shape signals "this signal is load-bearing".

Acceptance criteria

  • Extract a testable fetcher factory in src/lib.ts (e.g. crawlFetcher(timeoutMs, fetchImpl?) returning the (u) => fetch(u, { redirect: "manual", headers, signal }) function), and have crawlFallback in src/index.ts use it — same 20 s default, same headers, same manual-redirect mode.
  • New tests in test/lib.test.ts pin: (a) every call receives an AbortSignal; (b) a fetch whose response stalls rejects when the signal fires (short stand-in timeout); (c) a slow-drip body read through the returned response rejects on abort.
  • Ablation: the new tests fail if the factory stops attaching the signal.
  • npm test in pkgs/lab-ops-hub stays green (TS + Python suites).

Single-repo: lab-ops-hub / crawlFallback exist only in this repo (grep-verified against the opencode, openchamber and nix clones). Filed by new-issue-creator from the pr-diff-review REVIEW-FINDING recorded against PR #501 on 2026-10-21.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions