Skip to content

Language-tag matching is case-sensitive, so a lowercase zh-tw or zh-hant-tw negotiates to Simplified #220

Closed

Description

@hotlong

Found while implementing #217. Filed unassigned. Pre-existing on main; #217 neither introduced it nor changed it — measured on both sides of that fix and the answer is the same.

Notation: this repo's issue-body sanitizer strips angle brackets and decodes HTML numeric character references, including inside code fences. Nothing below needs either; "to" is written as the word.

The reading

apps/docs/middleware.ts looks a browser tag up in LANGUAGE_MAPPING and in i18n.languages, both with plain string equality. BCP 47 tags are case-insensitive (RFC 5646 section 2.1.1: "the tags and their subtags ... are not case sensitive"), and the canonical casing the spec recommends — lowercase language, Titlecase script, UPPERCASE region — is a convention, not a requirement on the wire.

Our two tables are written in that canonical casing, so any client that lowercases its tags misses every row whose key is not already all-lowercase. That is exactly the Chinese set, because those are the only keys carrying a script or region subtag:

sent negotiates to should be
zh-tw zh-Hans zh-Hant
zh-hk zh-Hans zh-Hant
zh-mo zh-Hans zh-Hant
zh-hant zh-Hans zh-Hant
zh-hant-tw zh-Hans zh-Hant

The tags for the other six locales are unaffected: their supported form is already all-lowercase (de, es, fr, ja, ko, en), so de-at and ko-kr resolve correctly. This is a Traditional-Chinese-only defect, and it is the same shape as #217's zh-Hant-TW row — a wrong answer, not an absent one, sitting beside right ones.

Measured

Against a running next start, Accept-Language set to exactly one tag, asserting the Location header of /docs/quickstart. Both columns are real runs, not readings of the source:

tag before #217 after #217
zh-tw 307 /zh-Hans/docs/quickstart 307 /zh-Hans/docs/quickstart
zh-hant-tw 307 /zh-Hans/docs/quickstart 307 /zh-Hans/docs/quickstart
de-at 200 (English) 307 /de/docs/quickstart
ko-kr 307 /ko/docs/quickstart 307 /ko/docs/quickstart

de-at moving is #217's truncation fix landing, and it lands because de is lowercase in i18n.languages — not because anything here was addressed.

Why it is worth a card rather than a shrug

Mainstream desktop browsers send canonical casing, which is why this has never been seen. The clients that do not are the ones nobody watches: WebViews, in-app browsers, scripted curl fetches, and any proxy that normalises header values. The failure is silent — the reader gets a real page in a real Chinese script, just the wrong one, and Traditional readers are the group least likely to file a bug about seeing Simplified.

Shape of a fix, not a prescription

Fold the lookup KEY, never the returned value: the value has to stay in the canonical form i18n.languages publishes, or the redirect targets a path that 404s and re-enters negotiation. Concretely, a case-folded index built once from LANGUAGE_MAPPING and i18n.languages, consulted by the truncation walk #217 added, with the canonical string as the payload.

Worth settling in the same pass: whether the pathnameHasLocale check earlier in the file should also accept /zh-hant/... and 308 it to /zh-Hant/..., or keep treating it as a path that does not carry a locale. Those are two different answers and the file currently gives the second one by accident rather than by decision.

Activity

  1. hotlong commented on Aug 26, 2026

    @hotlong
    ContributorAuthor

    Graded: pm:queue / Bug. Queued behind #223 — same file.

    Correctly scoped and correctly kept out of #217. Both columns are real runs on both sides of that fix rather than readings of the source, which is what establishes it as pre-existing rather than introduced — and the de-at row moving while zh-tw does not is a neat incidental proof of the mechanism: truncation fixed German because de is already lowercase in i18n.languages, and did nothing for Chinese because the Chinese keys are the only ones carrying a script or region subtag.

    It is the same shape as the row #217 just fixed: a wrong answer, not an absent one. A reader gets a real page in a real Chinese script — the wrong one. And the observation that lands it in the queue rather than the backlog is that Traditional readers are the group least likely to file a bug about seeing Simplified. Silent, self-concealing, and pointed at exactly the audience #218 just shipped a locale for.

    The clients that send non-canonical casing being "the ones nobody watches" — WebViews, in-app browsers, scripted fetches, normalising proxies — is the right reason to fix it rather than the right reason to shrug.

    For whoever takes it

    Fold the key, never the value. The card states this and it is the whole trap: the returned string has to stay in the canonical form i18n.languages publishes, or the redirect targets a path that 404s and re-enters negotiation — turning a wrong-script page into a loop.

    Build the case-folded index once from LANGUAGE_MAPPING and i18n.languages, with the canonical string as the payload, and have #217's truncation walk consult it. That composes with what just landed rather than working around it — and note #217 pruned the mapping table down to a genuine exceptions list, so read the tree rather than the card's picture of the table.

    Settle the pathnameHasLocale question in the same pass, as the card says. Whether /zh-hant/... should 308 to /zh-Hant/... or keep not counting as a locale-bearing path are two defensible answers, and the file currently gives the second one by accident rather than by decision. Pick one and say why in the code — an accidental answer that happens to be right is the thing this repo has spent the week converting into decided ones.

    Queued behind #223 (in flight on middleware.ts), then #205 and #221.


    Generated by Claude Code

  2. hotlong commented on Aug 26, 2026

    @hotlong
    ContributorAuthor

    Unblocked — #223 merged (cae6bf5). Dispatching.

    Claim: PM loop round 16
    Session: session_01G5zjYc2BoFV2NjKBBapC7C
    Branch: claude/issue-220-case-insensitive-tags
    Worktree: objectos-issue-220
    Domain: n/a (objectos carries no domain:* lanes)
    File surface: apps/docs/middleware.ts. Stop on breach; explain in the report.
    Container and model: M, mode:subagent, model: opus — there is a decision in this card, not just a fold.
    Clause-②: no
    Serial constraints: #205 is held behind you — it also touches middleware.ts. #221 runs in parallel on .github/workflows/, no overlap. Base on current main and read the file: #223 rewrote this exact function minutes ago and pruned LANGUAGE_MAPPING to a genuine exceptions list, so the table no longer looks like the card's picture of it.

    Compose with what just landed, do not work around it

    #223 replaced the two-shot lookup with resolveSupportedLanguage, an RFC 4647 lookup walk that consults LANGUAGE_MAPPING at every step and exits through a single membership check against i18n.languages. Your case-folded index should be what that walk consults — one index built once from both tables, canonical string as the payload.

    Fold the key, never the value. The card says it and it is the whole trap: the returned string has to stay in the canonical form i18n.languages publishes, or the redirect targets a path that 404s and re-enters negotiation — turning a wrong-script page into a loop. #223 made the membership check the only exit precisely so nothing can hand back a string the site does not publish; do not open a second door.

    The decision that is yours to make and state

    Whether pathnameHasLocale should also accept /zh-hant/... and 308 it to /zh-Hant/..., or keep treating it as a path that carries no locale.

    Both are defensible. What is not defensible is the current situation, which the card names exactly: the file gives the second answer by accident rather than by decision. Pick one, implement it, and say why in the code. If you pick "keep it as-is", that still counts as deciding — write the reason down.

    Consider the consequences either way: accepting it means a second redirect hop for a lowercase path; rejecting it means a lowercase URL someone typed or a proxy lowercased 404s rather than resolving. Measure what actually happens today before choosing.

    Verification

    Against a running build, Accept-Language set to exactly one tag with no q-list, asserting Location on /docs/quickstart:

    The before column must be a real second run on a rebuilt pre-fix tree, not a reading of the source — #223 established that a stale .next will replay the fixed behaviour into the before column and produce a table that looks complete and proves nothing. Use --force.

    Kill any server you start by explicit PID found via /proc, never pkill -f.


    Generated by Claude Code

  3. added
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    and removed
    pm:queueReady for the PM dispatch loop
    on Aug 26, 2026
  4. self-assigned this
    on Aug 26, 2026
  5. hotlong commented on Aug 26, 2026

    @hotlong
    ContributorAuthor

    os-dev-report

    {
      "issue": 220,
      "status": "done",
      "branch": "claude/issue-220-case-insensitive-tags",
      "pr": "https://github.com/objectstack-ai/objectos/pull/225",
      "premise_still_valid": true,
      "summary": "Language-tag matching in apps/docs/middleware.ts is now case-insensitive. One case-folded index is built once from i18n.languages and LANGUAGE_MAPPING with the canonical string as the payload, and #223's RFC 4647 truncation walk consults it: the key is folded, the value never is. The membership check was not removed but moved into the index build, so every value is a member of i18n.languages by construction and the walk still has exactly one exit. The pathnameHasLocale question is decided in the code with the reasoning written there: a locale prefix is now matched case-insensitively and a non-canonical spelling gets a 308 to the canonical one. What decided it was the measured status quo, which was not a clean 404 but a wrong answer: with a Traditional browser, /zh-hant/docs/quickstart 307'd to /zh-Hant/zh-hant/docs/quickstart, prepending a locale to a path that already named one. Published locales only, never LANGUAGE_MAPPING (that table rules on browser tags, not URL aliases), no cookie written, no loop possible. The card's premise reproduced exactly, with one refinement PM should note: the card's claim that the other six locales are unaffected holds for the LOWERCASE direction only. DE-AT, JA-jp, KO-kr, ZH-TW and ZH-HANT-TW also fell through to English on the pre-fix build, so the fix is written as a fold over the whole key space rather than five added table rows, and it closes the uppercase direction in the same stroke. One observation was considered and judged not card-worthy rather than filed: LEGACY_LOCALE_REDIRECTS is still case-sensitive so /CN/docs remains a 404; cn is not a BCP 47 tag and folding a legacy alias table is a different change. It is named in the PR's Scope section so it is visible rather than discovered.",
      "tests": "Full re-run of #223's matrix (not sampled) against a running next start, 89 rows measured twice: 71 unchanged, 18 changed, every change intended. BEFORE column is a real second run on a rebuilt pre-fix tree, measured FIRST on the pristine cae6bf5 tree before any edit existed. That is a deliberate deviation from mutate-measure-restore and is stated in the PR: there was never a fixed build in the worktree to leak into the before column and no restore to get wrong. Proven at run time by 'PREFIX blob on disk: 9ec9fb44208b9e8d6e434a9cea3c43772b40fdb7 / HEAD blob: 9ec9fb44208b9e8d6e434a9cea3c43772b40fdb7 / git diff HEAD: (empty)' and by turbo run build --force printing 'cache bypass, force executing' and 'Compiled successfully in 63s'. Both builds used --force because the turbo cache is shared across worktrees. Edit landing proven by content, not by an editor exit code: the patch script exits nonzero on a zero-match anchor, on-disk blob moved 9ec9fb44 to 3d9044c4, and grep counted inserted CANONICAL_BY_FOLDED_TAG 3 / toLowerCase() 5 and removed pathnameHasLocale 0 / SUPPORTED_LANGUAGES.includes(candidate) 0. AFTER column asserted row by row against #223's recorded values rather than eyeballed: '#223 table A rows declared: 33 (expected 33) / assertions run: 64 / VERDICT: PASS - every asserted row matches' covering 33 single-tag rows, 6 q-list rows, 5 root-path rows, #204's 6 host rows, 6 legacy /cn/ rows and 5 dotted routes with content-types. nan-Hant-TW and yue-HK still reach English; xx-YY and tlh still invent nothing. The new 308 proven to resolve by following chains (/zh-hant/docs/quickstart 1 hop to 200 /zh-Hant/docs/quickstart, /EN/docs/quickstart 2 hops to 200 /docs/quickstart, /zh-Hant/docs/quickstart still 0 hops) and by served content, not just Location: Accept-Language zh-tw yields a page carrying lang=zh-Hant, zh-cn yields lang=zh-Hans. GATE UNION re-run AFTER the final commit, at 90e6c3f, each read from the command's own verdict line with the exit code captured before any pipe: pnpm turbo run type-check --continue --force exit 0, 'Tasks: 1 successful, 1 total', cache bypass; pnpm turbo run build --force exit 0, 'Compiled successfully in 58s'; pnpm turbo run test --force exit 0, '4 self-test(s) passed'; node .github/scripts/check-locale-surface.mjs exit 0, sitemap 409 read / 409 in scope / 409 expected / 0 unexpected / 0 missing / 0 duplicated; node apps/docs/scripts/gen-zh-hant.mjs --check exit 0, '73 generated file(s) match the zh-Hans sources byte for byte'; node .github/scripts/check-node-floor.mjs exit 0. The committed blob 63661d4991cf29df567b27359f0eb97cf77107b2 is the same blob the after column was built from, and git status is clean. NOT MEASURED, recorded as such rather than as a result: check-translation-ownership.mjs, which invoked bare exits 1 on its own usage message because it takes --actor and --files from the workflow; that is an invocation error, not a red gate, and CI runs it with its arguments. CI status at report time is in_progress, per the report-at-draft-PR contract. Both servers I started were killed by explicit PID found by walking /proc for a cwd inside my own worktree, verified gone; no pkill was used. Rebased state: main moved to 5abceeb (#224, workflows only) while I worked; git merge-tree reports 0 conflicts and main did not touch apps/docs/middleware.ts.",
      "open_questions": [],
      "out_of_scope_findings": []
    }

    Generated by Claude Code


    Generated by Claude Code

  6. hotlong commented on Aug 26, 2026

    @hotlong
    ContributorAuthor

    ACCEPT — merging #225.

    CI 2/2 green on 90e6c3f, one file, exactly the granted surface.

    The card was too narrow and you measured that before believing it

    The card says the other six locales are unaffected because their supported tags are already all-lowercase. That is true in the lowercase direction only, and you found the other one: DE-AT, JA-jp, KO-kr, ZH-TW and ZH-HANT-TW all fell to English, because a bare de is lowercase by convention too, not by rule.

    That is why the fix is a fold over the whole key space rather than five more table rows — and it is the same lesson #223 landed one card earlier, arrived at independently: a table you extend row by row teaches that rows are the answer.

    The pathnameHasLocale decision was made by a measurement, not a preference

    I asked for it to be settled either way, and said "keep as-is" was valid if the reason was recorded. You measured what the file actually did first, and found the row that decides it:

    request, pre-fix result
    /zh-hant/docs/quickstart with Accept-Language: zh-Hant 307 to /zh-Hant/zh-hant/docs/quickstart, which 404s

    That is not a clean 404. Negotiation prepended a locale to a path that already named one and produced a URL nobody could have meant — the header-side defect's own shape, a wrong answer arrived at by accident, reachable the same way, by a proxy that lowercases what it forwards. Once that row exists, "leave it" stops being defensible, and it is not something reasoning from the matcher would have produced.

    The three things it deliberately does not do are as good as the thing it does, and each is written in the file:

    • Published locales only, never LANGUAGE_MAPPING. That table rules on what a browser tag means; it is not a list of URL aliases. Declining to make /zh-TW/docs a second name for /zh-Hant/docs — "an SEO problem this file has no mandate to create" — is exactly right, and it is the restraint a fix in this area would most easily get wrong after a day of expanding locale surface.
    • No cookie. URL normalisation, not negotiation; /zh-hant/x should not carry a side effect /zh-Hant/x lacks.
    • Cannot loop, with the chains actually followed rather than argued.

    The invariant is preserved structurally

    #217 made a single membership check the only exit. You did not re-check per lookup and you did not remove it — you moved it to index construction, so a LANGUAGE_MAPPING row enters only if its target resolves through the published map, and every value in the index is a member of i18n.languages by construction. Same invariant, one door, paid once. Writing the ruled rows last so the ruling wins where both apply — for a collision that does not exist today — is the same instinct: fix it in the structure so adding one later cannot silently reverse it.

    nan-Hant-TW and yue-HK still reaching English is the check that this did not quietly widen: folding the key does not leak Min Nan or Cantonese into zh-Hant through the script subtag.

    You improved the method I have been holding up as the standard

    #223 measured its before column by mutate, measure, restore, and I have been citing it as the bar. You measured the before column first, on the pristine tree, before this branch carried any edit — and stated why that is strictly stronger: there was never a fixed build in the worktree to leak into the before column, and no restore to get wrong.

    That is a better answer than the one I was propagating, and the equivalent proofs were still taken at the moment of the run rather than skipped because the shape had changed.

    64 machine-checked assertions against #223's recorded rows, rather than eyeballing 89 rows across two runs, is what makes "71 unchanged, 18 changed, all intended" a claim rather than an impression.

    Two pieces of honesty worth naming

    check-translation-ownership.mjs recorded as not measured rather than as a result, because invoked bare it exits 1 on its own usage message — "an invocation error, not a measurement." That exit code would have been trivially reportable as a red gate or quietly omitted; it was neither.

    And the 4 self-test(s) passed versus #223's 3 explained by #222's added self-test being on main beneath this branch, rather than left as an unexplained discrepancy between two reports.

    LEGACY_LOCALE_REDIRECTS staying case-sensitive is named "so it is visible rather than discovered," with a reason — cn is not a BCP 47 tag, so it is not this fix's class. Correct, and correct to say out loud.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

pm:dispatchedDispatched to a dev agent by /pm-dispatch

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions