Repository navigation
Description
Summary
PR #666 fixed the container validator returning the raw compatibility-mapped host spelling (#665) by measuring the host before its NFKC step and rebuilding the returned URL when that step changed the spelling. One spelling escapes that measurement entirely: U+212A KELVIN SIGN. The container judges http://K.example.com/ as host k.example.com but returns http://K.example.com/ verbatim, while the Worker validator returns http://k.example.com/. The two validators again hand back different URLs for the same input — the exact #665 divergence, for the one character whose fold happens before the new measurement point.
Evidence (verified on current main c1524b63, driving both real validators)
pkgs/lab-ops-hub/container/app.py:
- Host extraction folds the character before the fix(lab-ops-hub): return the mapped host for compatibility spellings in the container validator (#665) #666 measurement exists:
host = (u.hostname or "").lower()—urllib'shostnameproperty already lowercases, and Python's Unicode lowercase mapping for U+212A is ASCIIk. So the extracted host isk.example.comfrom the start. - PR fix(lab-ops-hub): return the mapped host for compatibility spellings in the container validator (#665) #666's
host_before_mapping = hostis captured from that already-folded host, sohost_was_mapped = host != host_before_mapping and host.isascii()isFalsefor a Kelvin-sign host: nothing downstream of extraction changes the spelling, and the public-DNS return path returnsrawunchanged. - Judging is unaffected (it sees the folded form):
http://K.localhost/is refused as local hostk.localhost. This is purely a return-form divergence.
Probe results:
container (Python) validate_crawl_url("http://K.example.com/") -> "http://K.example.com/" (raw returned)
container (Python) validate_crawl_url("http://K.com/") -> "http://K.com/"
container (Python) validate_crawl_url("http://Kexample.com:8080/p") -> "http://Kexample.com:8080/p"
container (Python) validate_crawl_url("http://%e2%84%aa.example.com/") -> "http://k.example.com/" (percent spelling correct)
Worker (TS) validateCrawlUrl("http://K.example.com/") -> "http://k.example.com/"
Worker (TS) validateCrawlUrl("http://K.com/") -> "http://k.com/"
Note the internal inconsistency: the percent-encoded spelling of the same host returns the judged form, the literal spelling does not. A mixed host also works (http://Ke.com/ -> http://ke.com/ in both), because the fullwidth character triggers the #666 rebuild — only a host whose sole fold is the Kelvin sign escapes.
Completeness: a brute-force scan of every Unicode codepoint through the extraction-fold → NFKC pipeline shows U+212A is the only character whose lowercase mapping lands on an NFKC-stable ASCII form, i.e. the only spelling this blind spot can ever hide. Every other cased compatibility character either keeps a non-ASCII judged form (the open #659 U-label question) or is changed again by NFKC, which #666's check does see.
Why it matters
Same principle as #665/#652/#660: the validated URL is the artifact both crawlers fetch, log, and compare, and the pair is maintained to return one canonical spelling for one input. Here the container records and fetches a spelling it never judged — the judged host k.example.com exists only inside the validator — and the container/Worker pair disagrees on the return for the same input, so behavior depends on which validator handled the crawl.
Acceptance criteria
-
validate_crawl_url("http://K.example.com/")returnshttp://k.example.com/; in general, a host whose spelling was folded at extraction time (before the NFKC step) also rebuilds the returned URL from the judged host. - The percent-encoded and mixed-spelling controls keep their current returns (
%e2%84%aa->k.example.com,Ke.com->ke.com), and spellings the pipeline does not change return exactly what they return today. - New cases pinned in
pkgs/lab-ops-hub/containerPython tests (Kelvin sign as a full label and inside a label, with a port/path), and the TS suite still passes unchanged.
Dedup / scope
Not #665 (compatibility mappings visible to the NFKC step — fixed by #666; this fold happens before that step, at urlparse extraction/lower-casing). Not #652 (UTS-46 ignored codepoints deleted by the translate step), not #659 (raw U-label returned for non-ASCII IDN targets — the Kelvin judged form is pure ASCII), not #660 (literal separator spellings). Found by PR-diff review of #666. Single-repo: the validator pair lives only in this repo's pkgs/lab-ops-hub (container + Worker); no copy exists in opencode/openchamber/nix.
Activity
Merge pull request #669 from developing-today/sweep/667-kelvin-sign-r…