Skip to content

lab-ops-hub: U+0345 inside an RTL label is certified — UTS-46 maps it to a Greek letter (bidi L), the container's NFKC mapping leaves it a combining mark #596

Closed

Description

@dezren39

Summary

PR #590 (fixes #583, commit 6bad7e15, merged as ead788e1) judges raw hosts in their mapped form in validate_crawl_url (pkgs/lab-ops-hub/container/app.py, current main ead788e1): the host is NFKC-normalized and lowercased before _judge_idna_label applies its bidi check. But the container's "mapped form" is NFKC, and UTS-46 mapping is not NFKC: U+0345 (COMBINING GREEK YPOGEGRAMMENI) is mapped by UTS-46 to U+03B9 (ι, bidi class L), while unicodedata.normalize("NFKC", "\u0345") returns it unchanged (its UnicodeData decomposition field is empty; the mapping lives in the special-casing data — Python's own "\u0345".casefold() does produce ι). The container therefore judges U+0345 as what it is before UTS-46 mapping — a combining mark, bidi NSM, legal mid-label in any context — while the Worker maps it to a Greek letter first and then applies bidi validity to the result. Inside an RTL label that flips the verdict: the mapped label is R + L + R, a bidi-rule violation.

PR #590's own comment in _judge_idna_label names U+0345 as a known residual, but only in the opposite direction ("U+0345, which UTS-46 maps to a letter, reads as a leading mark here" — the container over-refusing it at label start). The under-refusal inside RTL labels is not documented anywhere.

Evidence (verified 2026-10-22 on current main ead788e1, the merge commit of PR #590 itself)

Python validate_crawl_url imported from container/app.py at main vs the Worker's validateCrawlUrl from src/lib.ts under Bun; raw-form rows cross-checked with Node's new URL, which agrees with Bun on every raw case:

URL Container Worker
http://א\u0345א.com/ (U+0345 between two Hebrew letters) accepts refuses — url is not a valid URL
http://א\u0345.com/ (Hebrew letter + U+0345) accepts refuses
http://xn--jua51kba.com/ (A-label spelling of the first row) accepts refuses under Bun (Node's URL parser accepts this A-label spelling — engine-dependent, secondary)
Controls: http://ex\u0345ample.com/, http://α\u0345.com/ (LTR/Greek contexts) accepts accepts
Documented reverse residual: http://\u0345abc.com/ refuses (leading mark) accepts (maps to ιabc) — unchanged, already disclosed in PR #590's comment

Scope: this comes from a differential battery inserting every BMP codepoint U+0080–U+FFFF between two Hebrew letters (א<cp>א.com) through both real validators — U+0345 and U+FE12 (#593, a different root cause) are the only container-accept / Worker-refuse divergences in that position, and U+0345 produces none in Latin-mid, leading or trailing positions (leading is the documented reverse case above). A targeted scan confirms the mechanism is the mapping gap: "א\u0345א" after the container's NFKC+lowercase steps is still ['0x5d0', '0x345', '0x5d0'], so _judge_idna_label sees bidis [R, NSM, R] — no L, no mixing, trailing rule skips the NSM — while the Worker's mapped form is [R, L, R].

Why it matters

Same shape as #583: the two validators that are supposed to hold the same crawl-target policy (#441 defence in depth — the container is the component that actually fetches) disagree, here because the container's approximation of UTS-46 mapping (NFKC + lower()) silently diverges from UTS-46 mapping for a codepoint PR #590 itself flagged — and only the harmless direction of that divergence was documented. Any future codepoint whose UTS-46 mapping changes its bidi class or category relative to NFKC lands in the same hole; U+0345 is the one the current batteries can see.

Acceptance criteria

  • validate_crawl_url refuses a host whose UTS-46-mapped label violates the bidi rule because of the U+0345 → U+03B9 mapping — at minimum http://א\u0345א.com/ and http://א\u0345.com/ are refused with the same verdict as the Worker's validateCrawlUrl, in raw form and (under the repo's Bun differential) in the A-label spelling xn--jua51kba.
  • The LTR/Greek controls (ex\u0345ample.com, α\u0345.com) stay accepted; the documented leading-position over-refusal may stay as-is (Worker-accepts residual) or be closed by the same mapping fix, but must not silently change without a test pinning whichever behaviour is chosen.
  • New tests in pkgs/lab-ops-hub/test/test_container_validate.py (and the TS pin in test/lib.test.ts) cover U+0345 in an RTL label (refused) and in a Greek label (accepted).
  • No regression in the existing Python or TS suites.

Single-repo: validate_crawl_url / _judge_idna_label exist only in developing-today/code (git-grep verified in the opencode / openchamber / nix clones). Not an mcpx set. Deduped: searches for 0345 / ypogegrammeni return nothing; #593 (U+FE12 separator mapping) and #595 (missing AN rule) are different root causes in the same helper from the same PR; coordinate with #588 for the A-label spelling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions