Skip to content

lab-ops-hub: container validator appends the empty-query marker after the fragment — path?#frag returns as path#frag?, differing from the Worker #654

Closed

Description

@dezren39

Finding

PR #648 (#648, commit 7bd77d0937, merged as 93065f0d) added _empty_query_fragment_markers to preserve the empty ? / # markers WHATWG keeps (#639), but it re-attaches the markers as a suffix after geturl() at every rebuild site. When the query is empty and the fragment is non-empty, the ? therefore lands after the fragment instead of before the #:

Input Container returns (current main) Worker / WHATWG URL returns
http://example.com/path?#frag http://example.com/path#frag? http://example.com/path?#frag
http://example.com?#frag http://example.com/#frag? http://example.com/?#frag
http://example.com/a?#b?c http://example.com/a#b?c? http://example.com/a?#b?c
http://%65xample.com/path?#frag http://example.com/path#frag? http://example.com/path?#frag
http://8%2e8%2e8%2e8/path?#frag http://8.8.8.8/path#frag? http://8.8.8.8/path?#frag

Verified on current main by running validate_crawl_url from pkgs/lab-ops-hub/container/app.py (extracted fresh via gh api ...?ref=main) against the table above, cross-checked against Node's WHATWG URL serializer (the Worker's reference). The percent-decoded-host and canonical-IP rebuilds are affected too, because all three rebuild sites share the suffix pattern.

Evidence

Why it matters

This is exactly the class of divergence #639 set out to eliminate: the two crawlers record different URLs for the same accepted target. Worse than a dropped marker, the returned URL is a different URL — the fragment is corrupted (frag? instead of frag) and the present-but-empty query becomes absent — so anything keying on the recorded URL (dedup, redirect handling, downstream fetch of the recorded form) sees a different target than the one that was judged and that the Worker records.

Acceptance criteria

  • validate_crawl_url returns the WHATWG form for empty-query + non-empty-fragment spellings, e.g. http://example.com/path?#frag → http://example.com/path?#frag, including through the percent-decoded-host and canonical-IP rebuild paths.
  • The empty-fragment marker case stays correct (path?#, path#, path?x# unchanged) and a ? inside the fragment stays data (#frag?notquery unchanged).
  • New regression tests pin the ?#<non-empty fragment> combination in the Python suite (and the TS suite if the Worker pin is extended); the differential probe set includes empty-query × non-empty-fragment tails.
  • Existing Python / TS validator suites still pass; no verdict (accept/refuse) changes.

Activity

  1. added a commit that references this issue on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions