Repository navigation
Description
Summary
PR #590 (fixes #583, commit 6bad7e15, merged as ead788e1) added _judge_idna_label to the container crawl validator (pkgs/lab-ops-hub/container/app.py, current main ead788e1) to judge every non-ASCII label for IDNA/UTS-46 validity, including an explicit RFC 5893 bidi check. That check treats only bidi classes R/AL as right-to-left and has no rule for bidi class AN (Arabic Number) at all — beyond the pre-existing special case for the two BMP separators U+066B/U+066C in LTR labels (app.py, _judge_idna_label). Every BMP AN codepoint happens to be caught by another arm (the Cf check for U+0600–U+0605/U+06DD/U+0890/U+0891/U+08E2, the U+0660–U+0669 forbidden range from #574, the U+066B/U+066C case), so the gap is invisible in the BMP — but all 41 astral AN codepoints pass: the Hanifi Rohingya digits U+10D30–U+10D39 and the Rumi digits/numbers/fractions U+10E60–U+10E7E. In an LTR or neutral label the Worker's WHATWG domain-to-ASCII refuses them (an AN character is only valid inside an RTL label); the container certifies them as public DNS names.
Neither #583's battery nor PR #590's could see this: both inserted only codepoints U+0080–U+32FF, and every astral codepoint sits above that range.
Evidence (verified 2026-10-22 on current main ead788e1, the merge commit of PR #590 itself)
Python validate_crawl_url imported from container/app.py at main vs the Worker's validateCrawlUrl from src/lib.ts under Bun; the raw-form rows were additionally cross-checked with Node's new URL, which agrees with Bun on every raw case:
| URL | Container | Worker |
|---|---|---|
http://ex\U00010D30ample.com/ (Hanifi Rohingya digit in a Latin label) |
accepts | refuses — url is not a valid URL |
http://ex\U00010E60ample.com/ (Rumi digit in a Latin label) |
accepts | refuses |
http://\U00010D30abc.com/, http://abc\U00010D30.com/, http://münchen\U00010D30.de/ |
accepts | refuses |
http://\U00010D30.com/ (digit alone), http://1\U00010D30.com/, http://-\U00010D30.com/ |
accepts | refuses |
http://xn--example-eh36b.com/ (A-label spelling of the first row) |
accepts | refuses under Bun (Node's URL parser accepts this A-label spelling — engine-dependent, secondary) |
Controls: http://א\U00010D30א.com/, http://ا\U00010D30ا.com/, http://\U00010D00\U00010D30.com/ (digit in Hebrew / Arabic / its own Rohingya RTL label) |
accepts | accepts |
Differential battery: every codepoint U+10000–U+10FFFF inserted into ex<cp>ample.com (1,048,576 hosts) through both real validators — the container-accept / Worker-refuse pocket in the astral range is exactly these 41 AN codepoints (plus the U+FE12 separator case filed as #593). An exhaustive unicodedata scan confirms there are 63 AN codepoints in total; the other 22 are all BMP and all refused by the container today. Why the codec backstop does not catch them: "ex\U00010D30ample".encode("idna") succeeds — or, where the IDNA-2003 codec raises a bidi error, PR #590's re-judge treats AN as neither RTL nor a violation (its trailing rule only checks the last non-NSM class against R/AL/AN/EN for RTL labels and L/EN for LTR labels, and AN mid-label is never examined).
Why it matters
Same shape as #583/#574: the two validators that are supposed to hold the same crawl-target policy (#441 defence in depth — the container is the component that actually fetches) disagree, here on a whole bidi class that PR #590's explicit directionality check was written to cover. The Rohingya/Rumi digits are live scripts' digits, not unassigned codepoints, so this is not Unicode-version skew — both validators' tables contain them.
Acceptance criteria
_judge_idna_labelrefuses a label containing anANcharacter unless the label is a valid RTL label under the same rules the Worker applies — at minimum: AN in any label with a strong LTR character is refused, and AN in a label with no R/AL character is refused; the probed RTL controls above (Hebrew/Arabic/Rohingya contexts) stay accepted, matching the Worker.- All 41 codepoints U+10D30–U+10D39 and U+10E60–U+10E7E are refused in a Latin label, in raw and (under the repo's Bun differential) A-label form; the existing U+066B/U+066C-in-LTR behaviour is subsumed or preserved.
- New tests in
pkgs/lab-ops-hub/test/test_container_validate.py(and the TS pin intest/lib.test.ts) cover a Rohingya digit and a Rumi digit in a Latin label (refused) and in an RTL label (accepted). - No regression in the existing Python or TS suites.
Single-repo: validate_crawl_url / _judge_idna_label exist only in developing-today/code (git-grep verified in the opencode / openchamber / nix clones). Not an mcpx set. Deduped: searches for Rohingya / Rumi / 10D30 / 10E60 / "Arabic number" return nothing; #574 is closed and scoped to U+0660–U+0669, which stay refused via their own arm; coordinate with #588 for the A-label spelling, whose scope is decoded-label validity classes in general.
Activity
Merge pull request #604 from developing-today/sweep/595-an-bidi-rule