Skip to content

lab-ops-hub: container validator keeps the password colon when the password is empty (user:@ / :@) — returned URL differs from the Worker #651

Closed

Description

@dezren39

Problem

The container validator (pkgs/lab-ops-hub/container/app.py, validate_crawl_url) returns a URL whose userinfo keeps the password colon when the password is empty, while the Worker validator (pkgs/lab-ops-hub/src/lib.ts, WHATWG URL) serialises that colon away:

Input Container returns Worker returns
http://user:@example.com/ http://user:@example.com/ http://user@example.com/
http://:@example.com/ http://:@example.com/ http://example.com/
http://:@example.com:80/ http://:@example.com/ http://example.com/

Verified this run on main 93065f0d by driving both real validators over a 357-URL differential matrix plus a focused 23-URL probe; these were the only userinfo divergences. Controls that already agree and must not change: http://:pass@example.com/ (kept), http://::@example.com/ -> http://:%3A@example.com/ (the colon is password data), http://user::@example.com/ -> http://user:%3A@example.com/, http://user:pass@example.com/ (kept), and the bare empty userinfo http://@example.com/ (already dropped, #639).

Root cause

WHATWG serialises userinfo from the parsed username/password: the : separator is emitted only when the password is non-empty, and the whole @ is dropped when both are empty. The container instead round-trips the raw userinfo string:

  • _encode_userinfo (app.py ~line 895) re-adds ":" whenever a separator was present in the raw spelling (if sep:), regardless of whether the password is empty.
  • _url_with_encoded_components (app.py ~line 909) decides whether to keep the @ on the raw userinfo's truthiness (if userinfo), so ":" — a non-empty string with two empty components — survives as :@ even once the encoded form is empty.
  • _normalize_validated_url has the same raw-truthiness shape, though the final encoded-components rebuild is what reaches the caller.

#639 fixed exactly this class for the empty userinfo spelling (@ with nothing before it) but missed the colon-bearing spellings, because : and user: are non-empty strings.

Why it matters

The URL validate_crawl_url returns is the URL that is recorded and fetched (the module's standing #633/#639 contract: both crawlers record — and fetch — the same URL for the same target). Fetch behaviour already agrees — _split_userinfo (app.py ~line 979) treats :@ as carrying no credentials and user:@ exactly as user@ — so this is a returned-form divergence only: the same crawl target is recorded under two different URL strings depending on which crawler ran, defeating dedup/logging comparisons between the container and Worker paths.

Acceptance criteria

  • validate_crawl_url("http://user:@example.com/") returns http://user@example.com/ and validate_crawl_url("http://:@example.com/") returns http://example.com/, matching the Worker byte for byte (including with a default/non-default port present).
  • The controls above are unchanged: :pass@, ::@ -> :%3A@, user::@ -> user:%3A@, user:pass@, and bare @.
  • New cases in pkgs/lab-ops-hub/test/test_container_validate.py pin the returned form for the empty-password spellings (the existing tests assert acceptance only); Python suite and the TS suite (test/lib.test.ts) pass.
  • A differential run of both validators over userinfo spellings shows no remaining userinfo divergences.

Activity

  1. added a commit that references this issue on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions