Repository navigation
Description
Summary
The Lab Ops Hub container validator judges a host in its UTS-46/NFKC-mapped form but, for a host whose mapping is a compatibility mapping to ordinary ASCII letters/digits (e.g. fullwidth Latin), returns the caller's raw spelling — while the Worker validator returns the mapped form it judged. The two validators therefore hand back different URLs for the same input, and the URL the container records and fetches is not the spelling it judged.
Evidence (verified this run on current main 6da91105, driving both real validators)
pkgs/lab-ops-hub/container/app.py:
- Line 229: judging uses the mapped host —
host = unicodedata.normalize("NFKC", host).lower()(plus the separator translation at :230 and ignored-codepoint deletion at :231). The security checks do see through the spelling: fullwidthlocalhostis refused as local host localhost, and the fullwidth-digit IPv4 spelling is refused as 127.0.0.1. - Lines 426–431 (public-DNS return path): the URL is rebuilt from the judged host only
if percent_decoded or has_literal_separator— otherwiserawis returned unchanged. A fullwidth-letter host is neither, so the raw spelling survives.
Probe results:
container (Python) validate_crawl_url("http://example.com/") -> "http://example.com/" (raw fullwidth returned)
container (Python) validate_crawl_url("http://EXAMPLE.com/") -> "http://EXAMPLE.com/"
container (Python) validate_crawl_url("http://gpt2.example.com/") -> "http://gpt2.example.com/"
Worker (TS) validateCrawlUrl("http://example.com/") -> "http://example.com/" (mapped form returned)
Why it matters
The validated URL is the artifact both crawlers fetch, log, and compare; the container/Worker pair is maintained to return one canonical spelling for one input (the same principle behind #637's canonical IP-literal return, #652's ignored-codepoint return, and #660's separator return — #660's rebuild triggers only on has_literal_separator, so compatibility-mapped letters were left behind). Here the container fetches a URL whose host is only converted to the judged name implicitly, by the fetcher's stdlib IDNA-2003 nameprep — a different mapping table from the UTS-46 mapping the validator judged with, so for spellings where the two tables disagree the fetched host need not be the judged host. Returning the judged form removes the second, unjudged mapping from the path.
Acceptance criteria
-
validate_crawl_url("http://example.com/")returnshttp://example.com/— in general, whenever the judged (mapped) host differs from the caller's host spelling, the returned URL carries the judged host, as the percent-decoded (lab-ops-hub: container validator blanket-refuses percent-encoded hosts the Worker validator decodes and accepts — public spellings (%65xample.com, 8%2e8%2e8%2e8) diverge, including as redirect targets #529) and literal-separator (lab-ops-hub: literal U+3002/U+FF0E/U+FF61 root-dot spellings still return the fullwidth form from the container validator (PR #657 remainder) #660) paths already do. - Spellings whose host the mapping does not change return exactly what they return today (no gratuitous rebuilds; existing return-form tests keep passing).
- New cases pinned in
pkgs/lab-ops-hub/containerPython tests (fullwidth lowercase, fullwidth uppercase, fullwidth digit in a label), and the TS suite still passes unchanged.
Dedup / scope
Not #652 (UTS-46 ignored codepoints deleted from the judged host), not #659 (raw U-label returned for IDN targets vs the Worker's A-label), not #660 (literal separator spellings U+3002/U+FF0E/U+FF61), not #527 (judging of mapped spellings of blocked targets — that side works). This is the return-form remainder for compatibility mappings. Single-repo: the validator pair lives only in this repo's pkgs/lab-ops-hub (container + Worker); no copy exists in opencode/openchamber/nix.
Activity
Merge pull request #666 from developing-today/creator/665-fullwidth-m…