Skip to content

lab-ops-hub: container validator still allows bare 0x hosts (example.0x, 0x) that the Worker validator refuses #504

Closed

Description

@dezren39

Summary

PR #503 (fixes #502, merge eaf371c9, commit 6ff8f354) made the lab-ops-hub container crawler's validate_crawl_url refuse hosts whose final label is numeric, mirroring the Worker's WHATWG ends-in-a-number rule — but the refusal pattern requires at least one hex digit after a 0x prefix, so a bare 0x/0X final label still falls through to "a public DNS name" in the container validator while the Worker validator refuses every spelling:

Input TS Worker (validateCrawlUrl, src/lib.ts:129) Python container (validate_crawl_url, container/app.py)
http://example.0x/ BLOCK ("url is not a valid URL") ALLOW
http://example.0X/ BLOCK ALLOW
http://example.0x./ BLOCK ALLOW
http://0x/ BLOCK (WHATWG normalises to 0.0.0.0, non-public IPv4) ALLOW
http://0X/ BLOCK ALLOW

Evidence

  • Current main eaf371c9: pkgs/lab-ops-hub/container/app.py:65-66 —
    last_label = host.rsplit(".", 1)[-1]
    if re.fullmatch(r"0[xX][0-9a-fA-F]+|[0-9]+", last_label):
    The + means 0x with no following hex digit never matches, and a single-label 0x host takes the same fall-through (_inet_aton_ipv4("0x") returns None — socket.inet_aton("0x") raises "illegal IP address string", verified directly).
  • WHATWG's IPv4-number parser returns 0 for a bare 0x prefix (the digit sequence after the prefix is empty), so the host ends in a number: new URL("http://example.0x/") throws, and new URL("http://0x/") normalises the host to 0.0.0.0.
  • Both real validators were driven over the same inputs on current main this run (TS via node --experimental-strip-types importing src/lib.ts, Python by importing container/app.py): the five rows above diverge; the PR's claimed ~600-input differential parity evidently did not generate the empty-hex-digit case.
  • Controls verified identical on both sides (must not change): example.0x7f BLOCK/BLOCK, example.08 BLOCK/BLOCK, dead.beef ALLOW/ALLOW, example.0xg ALLOW/ALLOW, example.0x7fg ALLOW/ALLOW (a non-hex trailing character makes the label not an IPv4 number in WHATWG either, so it stays a DNS name).

Why it matters

The container validator is the defence-in-depth re-check on the component that actually fetches (#441), and #502's premise — where the two validators disagree, the effective policy is the weaker one — is exactly this class. A host spelling the URL grammar reserves for an IP literal (0x = 0.0.0.0, the non-public "this host" network as a single label) is reclassified as a public DNS name and handed to the fetcher's resolver instead of being refused up front with the validator's clean error. This is the residual of the change PR #503 landed to close #502, not a new policy question.

Acceptance criteria

  • validate_crawl_url refuses a host whose final label is a bare 0x/0X (no hex digits) when the host does not parse as an IPv4 address — including the single-label forms 0x/0X and the trailing-dot spelling example.0x..
  • Non-numeric-final-label names stay allowed on both sides: dead.beef, a1.beef, 123.example.com, and the non-hex-trailing controls example.0xg / example.0x7fg.
  • Regression tests in pkgs/lab-ops-hub/test/test_container_validate.py cover the bare-0x spellings (initial URL; redirect targets inherit via the same function), and npm test in pkgs/lab-ops-hub (TS + Python suites) stays green.
  • A TS/Python differential probe that includes the bare-0x cases shows no verdict divergence for this class.

Single-repo: lab-ops-hub exists only in developing-today/code (no validate_crawl_url/validateCrawlUrl copy in the opencode/openchamber/nix clones).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions