Skip to content

lab-ops-hub: container validator still allows DEL/C1-control, NBSP and WHATWG-forbidden (<>^|) hosts the Worker validator refuses #510

Closed

Description

@dezren39

Summary

PR #508 (fix for #507) added a whitespace check to the lab-ops-hub container crawler's validate_crawl_url (pkgs/lab-ops-hub/container/app.py:63 on main d4809cff):

if any(ord(c) <= 0x20 for c in host):
    raise ValueError(f"refusing to crawl host with whitespace {host!r}")

Its comment says "A space or control character in the host is not a DNS name", but ord(c) <= 0x20 covers only the C0 controls + space. The remaining control characters and the rest of the WHATWG forbidden-host set still fall through and are certified as "a public DNS name", while the Worker's validateCrawlUrl (pkgs/lab-ops-hub/src/lib.ts:129) refuses every one of them outright:

Host form TS Worker Python container
http://exam\x7fple.com/ (DEL) BLOCK ("url is not a valid URL") ALLOW
http://exam\x80ple.com/ … \x9f (C1 controls) BLOCK ALLOW
http://exam\xa0ple.com/ (NBSP) BLOCK ALLOW
http://exam<ple.com/, exam>ple, exam^ple, exam|ple BLOCK ALLOW

Controls, identical on both sides (must not change): ordinary DNS names, http://exam_ple.com/, and valid IDN hosts (e.g. examéple.com) are ALLOW/ALLOW — so the fix must not blanket-ban non-ASCII.

Evidence

  • PR: fix(lab-ops-hub): container validator refuses space hosts and IPvFuture literals #508, commit ebbeb8bca08895de704c720526b7842235590249, merged as d4809cffa77d9fb4695262445802a197ab1b6b4e.
  • Verified on current main d4809cff (git origin/main and gh api agree) by driving both real validators over the same battery: every printable ASCII character plus 0x7F–0xA0, é and a combining char, each embedded as http://exam<char>ple.com/ (TS via bun importing src/lib.ts, Python by importing container/app.py). The complete set of divergences was exactly: <, >, ^, |, 0x7F, 0x80–0x9F, 0xA0.
  • The DEL case reproduces lab-ops-hub: container validator accepts space hosts and IPvFuture literals the Worker validator refuses #507's exact lazy-failure shape: Python's own fetcher rejects it deep inside http.client, whose disallowed-character check is [\\x00-\\x20\\x7f] — i.e. http.client.HTTPConnection("exam\\x7fple.com").putrequest(...) raises InvalidURL: URL can't contain control characters — but the validator's new check stops one codepoint short of the fetcher's own set. The < > ^ | hosts are WHATWG forbidden host codepoints; no DNS resolver accepts them either, so those crawls also die below the validator instead of at it.

Why it matters

The container validator is the defence-in-depth re-check on the component that actually fetches (#441), and the parity premise of #468/#502/#505/#507 is that where the two validators disagree, the effective policy is the weaker one. PR #508 closed the space/C0 part of this class but left the validator's stated rule ("space or control character") under-implemented: a whole tail of not-a-DNS-name hosts is still certified as public DNS names, pushing the failure into the fetcher/resolver as a low-level error and leaving the guard's verdict meaningless for them.

Cross-repo note: this validator pair lives only in developing-today/code (git grep for validateCrawlUrl/validate_crawl_url in the opencode, openchamber and dezren39/nix clones finds no twin), so this is a single-repo issue.

Acceptance criteria

  • validate_crawl_url refuses a host containing DEL (0x7F), a C1 control (0x80–0x9F), NBSP (0xA0), or any of < > ^ |, up front with a clear validator error.
  • Controls unchanged: ordinary DNS names, exam_ple.com, valid IDN hosts, valid IPv6 literals and public targets are still accepted on both sides.
  • A regression test in pkgs/lab-ops-hub/test/test_container_validate.py covers each refused class, and fails with only container/app.py reverted (ablation).
  • The Python and TS validators agree on the full character battery above.

Activity

  1. added a commit that references this issue on Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions