Skip to content

lab-ops-hub: container validator returns the raw fullwidth/compatibility-mapped host spelling — Worker returns the UTS-46-mapped form it judged #665

Closed

Description

@dezren39

Summary

The Lab Ops Hub container validator judges a host in its UTS-46/NFKC-mapped form but, for a host whose mapping is a compatibility mapping to ordinary ASCII letters/digits (e.g. fullwidth Latin), returns the caller's raw spelling — while the Worker validator returns the mapped form it judged. The two validators therefore hand back different URLs for the same input, and the URL the container records and fetches is not the spelling it judged.

Evidence (verified this run on current main 6da91105, driving both real validators)

pkgs/lab-ops-hub/container/app.py:

  • Line 229: judging uses the mapped host — host = unicodedata.normalize("NFKC", host).lower() (plus the separator translation at :230 and ignored-codepoint deletion at :231). The security checks do see through the spelling: fullwidth localhost is refused as local host localhost, and the fullwidth-digit IPv4 spelling is refused as 127.0.0.1.
  • Lines 426–431 (public-DNS return path): the URL is rebuilt from the judged host only if percent_decoded or has_literal_separator — otherwise raw is returned unchanged. A fullwidth-letter host is neither, so the raw spelling survives.

Probe results:

container (Python)  validate_crawl_url("http://example.com/")      -> "http://example.com/"   (raw fullwidth returned)
container (Python)  validate_crawl_url("http://EXAMPLE.com/")      -> "http://EXAMPLE.com/"
container (Python)  validate_crawl_url("http://gpt2.example.com/")   -> "http://gpt2.example.com/"
Worker    (TS)      validateCrawlUrl("http://example.com/")        -> "http://example.com/"     (mapped form returned)

Why it matters

The validated URL is the artifact both crawlers fetch, log, and compare; the container/Worker pair is maintained to return one canonical spelling for one input (the same principle behind #637's canonical IP-literal return, #652's ignored-codepoint return, and #660's separator return — #660's rebuild triggers only on has_literal_separator, so compatibility-mapped letters were left behind). Here the container fetches a URL whose host is only converted to the judged name implicitly, by the fetcher's stdlib IDNA-2003 nameprep — a different mapping table from the UTS-46 mapping the validator judged with, so for spellings where the two tables disagree the fetched host need not be the judged host. Returning the judged form removes the second, unjudged mapping from the path.

Acceptance criteria

Dedup / scope

Not #652 (UTS-46 ignored codepoints deleted from the judged host), not #659 (raw U-label returned for IDN targets vs the Worker's A-label), not #660 (literal separator spellings U+3002/U+FF0E/U+FF61), not #527 (judging of mapped spellings of blocked targets — that side works). This is the return-form remainder for compatibility mappings. Single-repo: the validator pair lives only in this repo's pkgs/lab-ops-hub (container + Worker); no copy exists in opencode/openchamber/nix.

Activity

  1. added a commit that references this issue on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions