Skip to content

lab-ops-hub: PR #590's bidi check has no AN rule — Hanifi Rohingya and Rumi digits (41 astral AN codepoints) are certified in Latin labels the Worker refuses #595

Closed

Description

@dezren39

Summary

PR #590 (fixes #583, commit 6bad7e15, merged as ead788e1) added _judge_idna_label to the container crawl validator (pkgs/lab-ops-hub/container/app.py, current main ead788e1) to judge every non-ASCII label for IDNA/UTS-46 validity, including an explicit RFC 5893 bidi check. That check treats only bidi classes R/AL as right-to-left and has no rule for bidi class AN (Arabic Number) at all — beyond the pre-existing special case for the two BMP separators U+066B/U+066C in LTR labels (app.py, _judge_idna_label). Every BMP AN codepoint happens to be caught by another arm (the Cf check for U+0600–U+0605/U+06DD/U+0890/U+0891/U+08E2, the U+0660–U+0669 forbidden range from #574, the U+066B/U+066C case), so the gap is invisible in the BMP — but all 41 astral AN codepoints pass: the Hanifi Rohingya digits U+10D30–U+10D39 and the Rumi digits/numbers/fractions U+10E60–U+10E7E. In an LTR or neutral label the Worker's WHATWG domain-to-ASCII refuses them (an AN character is only valid inside an RTL label); the container certifies them as public DNS names.

Neither #583's battery nor PR #590's could see this: both inserted only codepoints U+0080–U+32FF, and every astral codepoint sits above that range.

Evidence (verified 2026-10-22 on current main ead788e1, the merge commit of PR #590 itself)

Python validate_crawl_url imported from container/app.py at main vs the Worker's validateCrawlUrl from src/lib.ts under Bun; the raw-form rows were additionally cross-checked with Node's new URL, which agrees with Bun on every raw case:

URL Container Worker
http://ex\U00010D30ample.com/ (Hanifi Rohingya digit in a Latin label) accepts refuses — url is not a valid URL
http://ex\U00010E60ample.com/ (Rumi digit in a Latin label) accepts refuses
http://\U00010D30abc.com/, http://abc\U00010D30.com/, http://münchen\U00010D30.de/ accepts refuses
http://\U00010D30.com/ (digit alone), http://1\U00010D30.com/, http://-\U00010D30.com/ accepts refuses
http://xn--example-eh36b.com/ (A-label spelling of the first row) accepts refuses under Bun (Node's URL parser accepts this A-label spelling — engine-dependent, secondary)
Controls: http://א\U00010D30א.com/, http://ا\U00010D30ا.com/, http://\U00010D00\U00010D30.com/ (digit in Hebrew / Arabic / its own Rohingya RTL label) accepts accepts

Differential battery: every codepoint U+10000–U+10FFFF inserted into ex<cp>ample.com (1,048,576 hosts) through both real validators — the container-accept / Worker-refuse pocket in the astral range is exactly these 41 AN codepoints (plus the U+FE12 separator case filed as #593). An exhaustive unicodedata scan confirms there are 63 AN codepoints in total; the other 22 are all BMP and all refused by the container today. Why the codec backstop does not catch them: "ex\U00010D30ample".encode("idna") succeeds — or, where the IDNA-2003 codec raises a bidi error, PR #590's re-judge treats AN as neither RTL nor a violation (its trailing rule only checks the last non-NSM class against R/AL/AN/EN for RTL labels and L/EN for LTR labels, and AN mid-label is never examined).

Why it matters

Same shape as #583/#574: the two validators that are supposed to hold the same crawl-target policy (#441 defence in depth — the container is the component that actually fetches) disagree, here on a whole bidi class that PR #590's explicit directionality check was written to cover. The Rohingya/Rumi digits are live scripts' digits, not unassigned codepoints, so this is not Unicode-version skew — both validators' tables contain them.

Acceptance criteria

  • _judge_idna_label refuses a label containing an AN character unless the label is a valid RTL label under the same rules the Worker applies — at minimum: AN in any label with a strong LTR character is refused, and AN in a label with no R/AL character is refused; the probed RTL controls above (Hebrew/Arabic/Rohingya contexts) stay accepted, matching the Worker.
  • All 41 codepoints U+10D30–U+10D39 and U+10E60–U+10E7E are refused in a Latin label, in raw and (under the repo's Bun differential) A-label form; the existing U+066B/U+066C-in-LTR behaviour is subsumed or preserved.
  • New tests in pkgs/lab-ops-hub/test/test_container_validate.py (and the TS pin in test/lib.test.ts) cover a Rohingya digit and a Rumi digit in a Latin label (refused) and in an RTL label (accepted).
  • No regression in the existing Python or TS suites.

Single-repo: validate_crawl_url / _judge_idna_label exist only in developing-today/code (git-grep verified in the opencode / openchamber / nix clones). Not an mcpx set. Deduped: searches for Rohingya / Rumi / 10D30 / 10E60 / "Arabic number" return nothing; #574 is closed and scoped to U+0660–U+0669, which stay refused via their own arm; coordinate with #588 for the A-label spelling, whose scope is decoded-label validity classes in general.

Activity

  1. added a commit that references this issue on Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions