Skip to content

lab-ops-hub: container validator's overlong-name check counts characters, not IDNA-encoded length — IDN labels/names over the RFC 1035 limits still pass in Python while the Worker refuses them #517

Closed

Description

@dezren39

Summary

PR #514 (fix for #513) added the RFC 1035 §2.3.4 length check to both lab-ops-hub crawl validators, but the container check in pkgs/lab-ops-hub/container/app.py counts characters of the raw host, while the Worker's check in pkgs/lab-ops-hub/src/lib.ts counts the punycode (IDNA-encoded) form, because u.hostname there is already punycode. For non-ASCII (IDN) hosts the two lengths differ, so the container validator still certifies IDN names that are over the limits — exactly the hosts #513's acceptance criteria said must be refused "with any label > 63 (encoded) characters", with both validators "matching each other's verdicts".

The Python code's own comment states the gap's safe half only: character length is a lower bound on the encoded length, so the check can never over-refuse. It can and does under-refuse: an IDN label whose characters fit in 63 but whose punycode exceeds 63 octets passes the container validator as "a public DNS name" and then fails deep inside the fetcher when the IDNA encoder rejects it — the lazy-failure shape #513/#502/#507 exist to eliminate. Python's own str.encode("idna") raises UnicodeError: label empty or too long for the probe label below.

Evidence (verified on current main bf642287, driving both real validators)

Container validate_crawl_url (pkgs/lab-ops-hub/container/app.py, overlong check at lines 69–74, comment at 60–68):

if ":" not in host and (
    len(host) > 253 or any(len(label) > 63 for label in host.split("."))
):

host here is urlparse(...).hostname.lower() — the raw spelling, never IDNA-encoded anywhere in the validator.

Probes (label = consecutive CJK codepoints U+4E00, U+4E01, …):

Host form Char length IDNA-encoded length TS Worker Python container
http://<42-char CJK label>.com/ label 42 label 65 BLOCK (overlong) ALLOW
http://<5 labels × 40 CJK chars>/ name 204 name 309 BLOCK (overlong) ALLOW

Controls that must stay allowed on both sides: a 63-char ASCII label, a 253-char ASCII name (both pinned by PR #514's own tests), valid IDN hosts within the encoded limits (e.g. examéple.com/, and a 40-char CJK label whose IDNA form is 61 chars — ALLOW/ALLOW today).

Why it matters

The container validator is the defence-in-depth re-check on the component that actually fetches (#441), and the parity premise of #468/#502/#505/#507/#510/#513 is that where the two validators disagree, the effective policy is the weaker one. For IDN hosts the container's length check is materially weaker: over-limit names are certified as public DNS names and die inside the fetcher's IDNA/resolver step with a low-level UnicodeError instead of the validator's clear up-front refusal.

Acceptance criteria

  • validate_crawl_url refuses, up front with the existing overlong-name error, a host whose IDNA-encoded label exceeds 63 octets or whose IDNA-encoded full name exceeds 253 characters (e.g. by judging the labels after per-label idna/punycode encoding, treating an encoding failure for length as a refusal).
  • Controls unchanged: 63-char ASCII label / 253-char ASCII name still accepted; valid IDN hosts within the encoded limits (including a 40-char CJK label, IDNA length 61) still accepted on both sides.
  • A regression test in pkgs/lab-ops-hub/test/test_container_validate.py covers the encoded-overlong label and encoded-overlong name cases plus an at-encoded-limit IDN control, and fails with only container/app.py's length check reverted to character counting (ablation).
  • npm test in pkgs/lab-ops-hub passes (TS + Python), and a differential probe over ASCII at/over-limit and IDN at/over-encoded-limit hosts shows TS/Python verdict parity.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions