Repository navigation
Description
Summary
PR #514 (fix for #513) added the RFC 1035 §2.3.4 length check to both lab-ops-hub crawl validators, but the container check in pkgs/lab-ops-hub/container/app.py counts characters of the raw host, while the Worker's check in pkgs/lab-ops-hub/src/lib.ts counts the punycode (IDNA-encoded) form, because u.hostname there is already punycode. For non-ASCII (IDN) hosts the two lengths differ, so the container validator still certifies IDN names that are over the limits — exactly the hosts #513's acceptance criteria said must be refused "with any label > 63 (encoded) characters", with both validators "matching each other's verdicts".
The Python code's own comment states the gap's safe half only: character length is a lower bound on the encoded length, so the check can never over-refuse. It can and does under-refuse: an IDN label whose characters fit in 63 but whose punycode exceeds 63 octets passes the container validator as "a public DNS name" and then fails deep inside the fetcher when the IDNA encoder rejects it — the lazy-failure shape #513/#502/#507 exist to eliminate. Python's own str.encode("idna") raises UnicodeError: label empty or too long for the probe label below.
Evidence (verified on current main bf642287, driving both real validators)
Container validate_crawl_url (pkgs/lab-ops-hub/container/app.py, overlong check at lines 69–74, comment at 60–68):
if ":" not in host and (
len(host) > 253 or any(len(label) > 63 for label in host.split("."))
):host here is urlparse(...).hostname.lower() — the raw spelling, never IDNA-encoded anywhere in the validator.
Probes (label = consecutive CJK codepoints U+4E00, U+4E01, …):
| Host form | Char length | IDNA-encoded length | TS Worker | Python container |
|---|---|---|---|---|
http://<42-char CJK label>.com/ |
label 42 | label 65 | BLOCK (overlong) | ALLOW |
http://<5 labels × 40 CJK chars>/ |
name 204 | name 309 | BLOCK (overlong) | ALLOW |
Controls that must stay allowed on both sides: a 63-char ASCII label, a 253-char ASCII name (both pinned by PR #514's own tests), valid IDN hosts within the encoded limits (e.g. examéple.com/, and a 40-char CJK label whose IDNA form is 61 chars — ALLOW/ALLOW today).
- PR: lab-ops-hub: refuse overlong DNS names in both crawl validators (#513) #514, commit
f7311bb1e609d196a8f521738096973361c3f7a7, merged asc7a00eee8323e11e0fe545bf6c80d7f026c190b5. - Origin issue lab-ops-hub: both crawl validators accept impossible DNS names — labels over 63 chars and names over 253 chars #513 (CLOSED by lab-ops-hub: refuse overlong DNS names in both crawl validators (#513) #514) acceptance: "Both validators refuse a host with any label > 63 (encoded) characters and a host whose full name is > 253 characters … matching each other's verdicts."
- Distinct from open lab-ops-hub: container validator still allows IDNA-disallowed Unicode hosts (space separators, bidi/format controls, U+FFFD) the Worker validator refuses #516, which is about IDNA-disallowed characters (space separators, bidi/format controls, U+FFFD), not encoded length.
- Cross-repo: the validator pair lives only in developing-today/code (the lab-ops-hub: container validator still allows DEL/C1-control, NBSP and WHATWG-forbidden (<>^|) hosts the Worker validator refuses #510/lab-ops-hub: both crawl validators accept impossible DNS names — labels over 63 chars and names over 253 chars #513 cross-repo checks found no twin in the opencode, openchamber or dezren39/nix clones), so this is a single-repo issue.
Why it matters
The container validator is the defence-in-depth re-check on the component that actually fetches (#441), and the parity premise of #468/#502/#505/#507/#510/#513 is that where the two validators disagree, the effective policy is the weaker one. For IDN hosts the container's length check is materially weaker: over-limit names are certified as public DNS names and die inside the fetcher's IDNA/resolver step with a low-level UnicodeError instead of the validator's clear up-front refusal.
Acceptance criteria
validate_crawl_urlrefuses, up front with the existing overlong-name error, a host whose IDNA-encoded label exceeds 63 octets or whose IDNA-encoded full name exceeds 253 characters (e.g. by judging the labels after per-labelidna/punycode encoding, treating an encoding failure for length as a refusal).- Controls unchanged: 63-char ASCII label / 253-char ASCII name still accepted; valid IDN hosts within the encoded limits (including a 40-char CJK label, IDNA length 61) still accepted on both sides.
- A regression test in
pkgs/lab-ops-hub/test/test_container_validate.pycovers the encoded-overlong label and encoded-overlong name cases plus an at-encoded-limit IDN control, and fails with onlycontainer/app.py's length check reverted to character counting (ablation). npm testinpkgs/lab-ops-hubpasses (TS + Python), and a differential probe over ASCII at/over-limit and IDN at/over-encoded-limit hosts shows TS/Python verdict parity.
Activity