Repository navigation
Fields/Headers take a list of Vec<(String, String)> to create, but probably should take Vec<(String, List<String>)> #821
Description
The new-entries function takes a list of (String, String) but the fields-get function returns a Vec(String)
Headers can have multiple values per the HTTP spec, so I think that new-entries should probably take Vec<(String, List<String>)>
Activity
lukewagner commentedon Mar 13, 2023
I wondered about this as well. One reason why perhaps not is: in the underlying protocols, iiuc, the fields are just a sequence of key/value pairs and thus if we cluster them by key, we're losing the protocol order which I could hypothetically imagine affecting some applications in corner cases. Also, for at least some common HTTP libraries (1,2), list-of-string-pairs is offered as the lower-level ground truth. Also, when you want to perform a higher-level mapping of a key to 0..N values, then the method you want is
fields-get(allowing the host impl to use a backing hash-table (e.g.)). Thus, I thinklist<tuple<string,list<string>>>sits in a sort of no-man's land between the lower-level sequence (that I think is commonly used in the representation) and the higher-level map.But that's just my understanding based on a partial knowledge of this space, so I'm happy to hear other thoughts too.
pimterry commentedon Mar 13, 2023
This is notable because quite a few services (notably Cloudflare) use the details of this header ordering & header key case (
Connectionvsconnection) for HTTP fingerprinting to detect types of client, and in many configurations they'll automatically block HTTP traffic based on that alone as a bot detection heuristic. Affects HTTP proxies, web crawlers, scraping tools, uncommon web browser implementations, etc etc - big chunks of the web become inaccessible via HTTP if you can't precisely control raw header order & casing.Not something that most normal HTTP API clients need to care about much, but imo it's definitely worth having low-level
tuple<string, string>[]APIs available everywhere to support these cases (for example Node.js has aheadersmap andrawHeadersarray available on all HTTP messages, andwriteHead(status, headers)accepts headers in both KV map & string array formats).brendandburns commentedon Mar 13, 2023
@lukewagner my biggest feedback is that there is inconsistency between the way that you create a new headers object (string, string) and the value that is returned when you call get (vec). And then of course there's also an 'append' method.
I don't feel strongly, multi-header is definitely an edge case, but we should be consistent at least.
lukewagner commentedon Mar 13, 2023
Sorry, I think I'm missing your point. As I'm reading it,
new-fieldstakes alist<tuple<string,string>>andfields-entriesreturns the same type, so we're symmetric in the case of bulk construction/consumption. The other methods are for finer-grained queries/mutations that don't require touching the complete list of fields.pchickey commentedon Mar 13, 2023
I believe his argument is that
new-fieldsshould take alist<tuple<string,list<string>>>, a design change I am also in favor oflukewagner commentedon Mar 13, 2023
Wouldn't that also lead to the same loss in control over ordering as if
fields-entriesreturned alist<tuple<string,list<string>>>? There's also a related question of whether the caller ofnew-fieldswould be allowed to pass the same key multiple times in the outer list, and whether that's enforced and, if not, what happens (including in chaining scenarios). (As a side note: it's unfortunate we don't have amaptype, which is complicated for a lot of the same reasons.)brendandburns commentedon Mar 14, 2023
I took a random look at a common HTTP library (https://square.github.io/okhttp/4.x/okhttp/okhttp3/-headers/) and it looks like the way they solved this is that they have both a
field-entry(which returns the first value)field-entries(which returns the list of values) as a way of making the API more explicit.lukewagner commentedon Mar 14, 2023
Starting with what we have proposed now:
fields-entries: func(fields: fields) -> list<tuple<string,string>>fields-get: func(fields: fields, name: string) -> list<string>I'm guessing you're not asking for a variation on the first one that just returns the first value (hard to imagine that one being useful), but, rather, a variation on the second that just returns the first value for a given
name(or none), so perhaps that suggests a name like:fields-get-first: func(fields: fields, name: string) -> option<string>PiotrSikora commentedon Mar 15, 2023
Usually, such API is a result of fixing the issue of returning only the first value, while retaining backward compatibility at the API level, i.e. it's not a proper solution for a new API.
brendandburns commentedon Mar 15, 2023
I think that the fundamental issue is that most people's model of HTTP headers is a hash-map. This isn't correct, but it is the general mental model.
If we're going to push people towards the "list of tuples" mental model, then we need to be consistent, so I would prefer
fields-entries: func(fields: fields) -> list<tuple<string, string>>fields-get: func(fields: fields, name: name, ix: int32) -> option<string>fields-count: func(fields: fields) -> int32I think the challenge with
fields-get: func(fields: fields, name: string) -> list<string>paints a picture where it really looks like the structure is a hashmap.Additionally given that the common (by far) case is a single
(string, string)tuple for each header name, returning a list makes the programmer do extra work that is dissonant.brendandburns commentedon Mar 15, 2023
fwiw golang also separates
Header::Get(get the first value) fromHeader::Values(get all values)https://pkg.go.dev/net/http#Header
On the other hand, dotnet only returns the list:
https://learn.microsoft.com/en-us/dotnet/api/system.net.http.headers.httpheaders.getvalues?view=net-8.0#system-net-http-headers-httpheaders-getvalues(system-string)
So maybe it doesn't matter :)
dicej commentedon Jun 8, 2023
Seems to me that
new-fieldsshould mirrorfields-entriesand accept alist<tuple<string,list<u8>>>.lukewagner commentedon Jun 12, 2023
Ah yes, I think this was an omission in WebAssembly/wasi-http#29. I'll file a follow-up fix.
lukewagner commentedon Jun 12, 2023
WebAssembly/wasi-http#42
sw17ch commentedon Oct 8, 2026
The WASI HTTP champions looked at this in triage. We believe that the conclusions of this discussion are now reflected in the specification. Please open new issues against the state of the specification today.