wasi-http 0.3: instance reuse means middleware world cannot implement CDN-Loop

@cfallin

The wasi-http 0.3 middleware world, as well as the work done to make instance reuse the default, are both features I'm strongly in favor of. The trouble comes with the intersection of those features, which I think breaks some assumptions. @cfallin first brought this issue up in #918.

CDN-Loop is a header that we expect many wasi-http embedders to require in their implementations of wasi-http middleware worlds. In short, the implementation will read the CDN-Loop header out of the incoming request, parse out the values specific to their CDN, possibly choose to block the request if those values exceed some limit, and otherwise the implementation will hand the request to a wasi-http middleware world's exported handler. Then, the implementation of the wasi-http middleware's imported handler needs to take the request and add a CDN-Loop header with all of the values from the inciting request's header, plus an added (on first hop) or modified (when appending a hop) value specific to the CDN. This is straightforward to implement in wasi-http 0.2, and indeed many implementations do so today.

However, with instance reuse, once there are multiple simultaneous requests passed to the exported handler, it is no longer possible for an implementation to associate a call of middleware's imported handler with the inciting request of middleware's exported handler. This ambiguity makes impossible to implement CDN-Loop correctly, and if implemented incorrectly CDN-Loop could end up either incorrectly blocking benign requests, or failing to block requests that should be blocked, neither of which a CDN can tolerate.

Inside any correct middleware component, with instance reuse or not, there is an association between an incoming request and the outgoing request, in order to turn the outgoing request's response to the incoming request's response. Instance reuse, and the current shape of the middleware wit world, makes it impossible for the implementation of the middleware world to maintain that association. Host magic to solve this doesn't exist, and even if it did, we want this exact sort of thing to be implementable with virtualization anyway.

CDN-Loop is just one example in the broader category of problems caused by loosing association between the inbound request and outbound request in a middleware or proxy. This problem presents in other ways specific to the embeddings that @cfallin and I maintain for F5, as well, but CDN-Loop is a representative example that exists across many vendors, so I figured its the best way to discuss it in the standard.

@pchickey

Thanks @pchickey for writing this up -- Pat and I have discussed this and a few potential solutions, starting from the discussion in #918. A few thoughts:

I think that this issue is separable from the logging issue in wasi-http 0.3: request and response should have a tracing identifier #918. While the latter -- incoming-request attribution for logged messages -- is a "nice-to-have correlation", this issue is a fundamental capability-model issue.
The cleanest rendering IMHO is to make the plumbing explicit in the dataflow. I had proposed something pretty ugly in wasi-http 0.3: request and response should have a tracing identifier #918 with a parent-request: option<borrow<request>> on a request constructor. Instead of that, I think maybe what we could have as a general shape is a factory pattern: the outgoing request method is no longer a freestanding function import, but a method on a resource, and that resource (outgoing-request-factory) is a capability ("the ability to make outgoing requests") that is plumbed into the entrypoint somehow.
- Within the guest, the incoming request associated with some dynamic scope could be held in (guest-side) thread-local storage or other implicit interpreter/runtime context. There is no conflict here wrt expected APIs in common guest languages IMHO: syscall or library interfaces of various sorts already force various kinds of impedance matching, and there should be an expectation in a capabilities-based system that one needs to wire through capabilities to provide the needed services.
- The main downside of the option-of-parent-request, that one would forget to associate the parent, is no longer an issue if we take the pure capabilities approach: one needs, by the static type system, to have the factory that the host can associate with some request context.
- This is also nicely virtualizable, because it's purely in the WIT type system as ordinary interfaces, with no magic.
- The main downside is that it results in a kind of non-compositional entrypoint: the naive dependency-injection design here is to have handle(outgoing-request-factory: ..., logger-factory: ..., other-side-service-factory: ...)
  - An antidote to that is to have a root-request-context resource, and use interface composition ("mixins") to request the specific factories. So the specified entry point for a wasi-http handler is handle(incoming-request, root-context), and then one does root-context.outgoing-request-factory() to get the capability to make outgoing requests. I strongly favor this design so far.
An alternative is to wire through the association on the host side, but (i) ambient dynamic state is IMHO an unfortunate design choice in a system that otherwise strives to be a capabilities system, and (ii) as @alexcrichton notes here there are complications in trying to build that plumbing.

@pchickey

I talked about this with @pchickey today a bit and wanted to write down some conclusions we reached.

The main thing was that I talked to @lukewagner about my comment here and we've concluded that it's probably what we should do to solve this. Specifically Wasmtime is already maintaining an async call stack of sorts and coupled with the ability to learn the "id" of a starting task I think it would solve this. Wasmtime can, for example, provide the ability to create an async task, not yet started, and then learn some sort of identifier from it. From this state the task can be started. Subsequently when wasm calls the host it should be possible for Wasmtime to deduce what the "root" async task is, and then the host can correlate these two events to provide a signal of what to do. Effectively, using async tasks, the host can correlate outgoing requests to whatever incoming request originated the task.

In talking with @pchickey we concluded that this would likely solve the issue at hand. There's some niche cases where it doesn't always work, but this is something we concluded is like inevitable regardless of the design. For well-behaved guests that don't intentionally go out of their way it should be able to provide this correlation.

With this the hope is then to be able to use this correlation to avoid needing to change anything related to the component model or WASIp3 WITs or such. I'll talk a bit more about this in the WASI meeting tomorrow and with some folks in the morning, but I wanted to be sure to at least leave a high-level summary comment before doing so to avoid completely blind-siding folks at least.

I've attempted to further summarize, and document, the outcomes of all discussions in #920. I've currently scheduled that to close this upon merge, so please take a look!

@alexcrichton

Thanks @alexcrichton. I don't have an invite to the WASI subgroup and didn't realize this would be discussed there, but I think it would be nice to see explicit commentary on the capabilities-based design noted above, especially in a longer-term context. It sounds like there is not appetite for creating explicit plumbing (in a canonical capabilities-system way) whereby the handler is given a capability to make outbound requests in its context? If not, why not? Implicit, ambient state, tied to tasks, seems to be strictly more complex to me. I understand pragmatism around getting near-term launches unblocked but I also want to make sure my ideas and concerns are not ignored in a longer-term design context.

The argument against plumbing through the association between inbound and outbound request via capabilities (i.e. plumbing through a resource from one to the other) is that while that can solve this particular problem where both end of the association exist in the same packge, it binds the invariant permanently to that package's interface (making it unsuited for reuse in other packages), and also cannot express invariants created by another package that uses another interface. Its just not a composition mode that was thought about when wit was designed, so there aren't mechanisms to do it currently - it would perhaps take something like an effect system in order to express these invariants across package boundaries, and may be tricky to implement in a way that doesn't break separate compilation of Canonical ABI imports/exports.

I agree that having to solve this issue by plumbing through a capability implicitly on the task abstraction is not ideal, but its the best mechanism we have a reasonable path to today. Long term its probably worth considering how the design of the component model's type system can express this differently.

Thanks. I suspect there is still a way that preserves separate compilation and composition over multiple packages. Consider:

The handle entry (or commandline main, or any world's entry point) takes a root-capability; that is a core WASI concept.
Any package P that today defines ambient-action(...) -> ... as a free function in an interface (e.g.: outbound HTTP requests) does the following:
- puts that method instead on an interface action-cap { action(...) -> ... }
- defines a ambient-action-capability(root: borrow<root-capability>) -> action-cap free function

That's a one-for-one trade in free toplevel function (ambient-action -> ambient-action-capability), so we still get the linker's ability to late-bind and preserve separate compilation. And the toplevel for any world does not need to know about the various system capabilities -- we get the N-to-M binding without having to make all aware of all. Basically, it inherits all of the characteristics of the existing situation, because we replace a toplevel function with a toplevel function -- the only difference is that we thread through root-capability. What do you think?

Concretely, one possible shape (but not the only one), rendered in wit:

package wasi:capability;

resource root-capability {}

# ---

package wasi:http;

use wasi:capabillity/root-capability;

interface handler {
  handle: async func(root: borrow<root-capability>, request: request) -> result<response, error-code>;
}

resource client {
  get: static func(root: borrow<root-capabililty>) -> client; # maybe result<>?
  send: async func(root: borrow<root-capability>, request: request) -> result<response, error-code>;
}

Note that:

Some other API that also needs to be "scoped" to the particular handle invocation could implement its own equivalent of the client resource, with no involvement from wasi:http, either at the type level or with extra params -- everything is hung off of the root-capability;
Separate compilation is preserved, because of the above;
The handle function on handler (incoming entry point) and send signature on client are symmetric (pass root cap in, pass root cap out).

One could summarize this approach as "thread a context handle through, and make actions methods on instances created from context, rather than freestanding functions".

@cfallin

I stand corrected, I hadn't thought of @cfallin's way to shape this so that a capability works with separate compilation.

I guess the biggest thing to consider with regard to @cfallin's proposal is where in the wasi interfaces root: borrow<root-capability> isn't appropriate as the first argument. get-environment() -> list<(string, string)> probably not because its supposed to be the same for the life of the instance, but what about get-stdout, or other logging facilities (as we discovered with #918)? I'm not sure where to start or stop applying that approach, which doesn't necessarily count against the approach but against my having thought of a rule for where to draw the line...

@cfallin

Ah apologies @cfallin this all happened pretty quickly, but regardless definitely happy to follow-up here.

Your example WIT is what I would envision as well for passing around capabilities, more-or-less an explicit argument in various locations. The problems I personally see with this sort of approach are:

Libraries in languages generally aren't architected for this. For example we wanted to ideally provide stdin/stdout/stderr as capabilities to the main function of a component, but that's just not how any standard library is really organized. For the context of HTTP this would mean that reqwest, or any other library that wants to support WASI, would have to have a WASI-specific API not present on any other platform to thread around this capability and/or object. Overall I feel that we've historically found that it's basically not worth the complexity budget to try to get ecosystems to integrate in WASI-specific ways and it's much easier to integrate as-is into ecosystems.
Assuming we did in fact do all the integration and plumbing everywhere, this can quickly become a related problem when the next capability needs to be transferred. For example wasi-otel may want a context of some form transferred around in addition to the wasi-http capability. This might instead nudge the host to have a sort of "catch all" capability which represents everything that needs to get passed around. This, to me, feels like it's going back to the world of "every interface takes a wasi:io object somewhere" where many interface would take this catch-all capability (to avoid having unique capabilities that everyone's passing around).
In the limit, to me this sounds very similar to a free-standing get-the-capability: func() -> own<thing> function. That'd be one-per-capability and per-interface, in theory, and would be easy to integrate into existing libraries. This sort of design though ends up being basically the same as using the call-stack and that's probably how the host would implement it.

Overall I feel that explicitly threading capabilities, while conceptually nice, generally doesn't work well for foundational primitives. For example standard I/O, making HTTP requests which many preexisting libraries already do, and tracing spans are examples of pretty foundational things where it'd be "cleanest" to thread objects around but AFAIK most languages don't do that. Using the async call stack is indeed just an approximation of what capabilities would allow (some examples in #920), but I feel at least personally that it's the best tradeoff of practicality and accuracy.

Libraries in languages generally aren't architected for this

I touched on this above already but I do not see this as changing user-visible APIs at all. Rather I would suspect that the guest would use some sort of dynamic context inside guest space to hold the root capability. That could either be guest-side TLS or state attached to an interpreter/language context. If you like, imagine a get_root_capability / set_root_capability facility somewhere inside wasi-libc or wstd or whatever that my-special-api-library uses to get the root cap for the "current execution".

That may seem like a little bit of sleight-of-hand vs. holding it "on the outside" (it's TLS either way!) but there are two big advantages: (i) it reduces the complexity of the host-side VM and required builtins, and I believe CM already has a problem with an unwieldy implementation complexity; (ii) it's virtualizable. Putting it in the WIT types and making context a real resource handle lets one reason about it, use it, pass it around like anything else, without special ambient state that requires its own primitives to access and update.

Assuming we did in fact do all the integration and plumbing everywhere, this can quickly become a related problem when the next capability needs to be transferred. For example wasi-otel may want a context of some form transferred around in addition to the wasi-http capability. This might instead nudge the host to have a sort of "catch all" capability which represents everything that needs to get passed around.

As noted above, wasi-http doesn't need to know about wasi-otel, and vice-versa. The "catch all" capability that you hypothesize is in fact root-capability, no? There's no need to transfer around a bunch of capabilities.

In the limit, to me this sounds very similar to a free-standing get-the-capability: func() -> own function. That'd be one-per-capability and per-interface, in theory, and would be easy to integrate into existing libraries. This sort of design though ends up being basically the same as using the call-stack and that's probably how the host would implement it.

Yes, exactly! :-) The key is that it's on the guest side, using existing facilities for guest-side state, and at the wit interface level (but not the user-visible level), things are explicit. That keeps the host simple(r) and keeps things virtualizable.

Ah sorry I missed your note of that, but I believe this sort of scheme still suffers from the inability to practically virtualize as well as a murky definition of root-capability on the host.

For virtualization this sounds a lot like wasi:io where we ended up concluding for WASIp2 that virtualizing more-or-less wasn't possible because if you virtualize one thing you have to virtualize the entire world due to the pervasive dependency on wasi:io. In this scheme the definintion of root-capability suffers the same problem -- if you virtualize wasi:http/handler then you also have to virtualize anything else that works with root-capability or similar.

For a murky definition, I'm not sure what we would do on the host for something like root-capability. One option would be like HashMap<TypeId, Box<Any>> to represent pretty much anything, and another option would be a sort of "trusted integer" which the host keys on (similar to the exported task ID if using the async call graph). If the latter, this is pretty close to the six-to-one-half-dozen or the other here.

Personally I'd say that adding this to WIT does not keep the hosts simpler. Hosts already have to maintain an async call graph and already have the knowledge of root tasks and such. Exposing this in Wasmtime is expected to be almost trivial and mostly just some bikeshedding. Alternatively all WIT interfaces need to be rethought and have borrow<root-capability> injected anywhere the host might want to handle things like observability. This doesn't just affect WASI WITs but also all downstream wits for embedders as well. Arguably something like wasi:sockets may want telemetry depending on the embedding as well, but on the other hand it seems weird to put it there.

Overall I see this need to attribute import-calls-to-export-calls as something that's so pervasive that solving it in WIT is the wrong layer of abstraction. It's sort of like how we could build an async system in WASIp2, but it was always known to be objectively the wrong place to do so.

From a capability-based security perspective: at the wasm level (as opposed to language level) our unit of isolation here is the component instance. That means that we have to assume that everything inside the unit is "colluding" (via shared instance-wide memory and table state) which means that trying to enforce something finer-grained at the wasm level isn't buying us anything. So for example, if, as a host (or parent component), I give a component instance ci 3 root-capabilitys, I have to assume that any import call made by ci can be passed any of those 3 root-capabilitys and that I have no way to know whether the given root-capability was "right" or "wrong" because the caller is a soup of core wasm; I have to assume it was right. If I need a finer degree of isolation, I need to create finer units of isolation (= component instances).

Importantly, from this capability perspective, I believe the async-call-stack-based approach is no different: every async call into ci creates a fresh "task" which you can indeed view as a capability (it can't be forged, and its meaning is set and interpreted by the host or, one day, parent component), and any import call that ci makes can only be given one of these tasks. And as above, I can't know whether "the calling task" was "right" or "wrong". The main difference is just that this capability is threaded implicitly through calls and thread creation by the runtime using machinery it already needed for several other purposes. There's also a temporary limitation which is that the guest can't easily control "the current task", but that could be fixed by an (I think) simple addition like the one sketched in CM/#659.

Given this more-theoretical capability equivalence, deciding between the approaches I think comes down to a more pragmatic question of ergonomics and ease of use which, for the reasons Alex mentioned, I think favors the async call stack approach. Also, all else being equal: the simpler and more readable our WIT interfaces are, the better, and I think plumbing root-capabilitys everywhere is a non-negligible regression; it's an abstract concept that you need a lot of context to understand, and if the goal is that it's plumbed by out-of-sight machinery, now WIT interfaces can't simply be defined and called by regular developers. If all this paid for itself by a clearly stronger security posture, that'd be worth discussing, but, because of the above, I don't believe that it does.

if you virtualize wasi:http/handler then you also have to virtualize anything else that works with root-capability or similar.

For a murky definition, I'm not sure what we would do on the host for something like root-capability. One option would be like HashMap<TypeId, Box> to represent pretty much anything, and another option would be a sort of "trusted integer" which the host keys on (similar to the exported task ID if using the async call graph). If the latter, this is pretty close to the six-to-one-half-dozen or the other here.

Yes, I had expected (sorry for not sketching details) that the root-capability would provide some sort of index so that a guest-space-implemented virtualized component for just one API could key its resources off of that.

I agree that this is pretty close to "trusted CM builtin to provide the current task ID" if accessible from Wasm; it sounds like your proposal above was to make this accessible only on the host side? That was one of my roots of concern about virtualizability. If you see root-capability as a reified, unforgeable task ID, I guess they're pretty similar.

Separately, there is a category of use that "actual task ID" does not cover that "task ID as unforgeable handle" does: when the guest wants to do smart (clever? too clever?) things managing its own contexts. For example do we have a guarantee that in all cases, for all guest task-scheduler implementations, we will have a one-to-one mapping between CM async tasks and the identity we wish to use for outbound calls? (We certainly have at least one CM async task per toplevel handle() entry, so maybe that's the case, but it's worth noting that the explicit plumbing is strictly more expressive here.)

From a capability-based security perspective: at the wasm level (as opposed to language level) our unit of isolation here is the component instance. That means that we have to assume that everything inside the unit is "colluding" (via shared instance-wide memory and table state) which means that trying to enforce something finer-grained at the wasm level isn't buying us anything.

Yes, to be clear, I am not proposing this from a security perspective. I fully understand that instance reuse already throws out any hope of isolation. The proposal is to link up inbound and outbound calls, which our platform (for one) absolutely requires, and I believe yours does too (see Pat's initial writeup).

I agree this is fundamentally an ergonomics question, with the caveat just noted above: the explicit plumbing is more expressive if the guest manages interior async tasks in a clever way. To note it explicitly: "async task as an implicit identity" is more or less imposing a requirement like a browser's main-thread constraint: outbound calls to host resources need to happen within a particular async context. (EDIT: well, WebAssembly/component-model#659 could in theory allow one to set the current task before calling out; but, possible today with plumbing vs. hypothesized future CM feature is certainly a tradeoff point.)

Virtualization and handling internal guest schedulers is indeed a problem yeah when the host is doing attribution via the async call stack. This is touched on a bit with #920, but also, yes, that's also the motivation for WebAssembly/component-model#659 -- to provide more async call stack backtraces in the host for various events. It's expected that languages like JS and Go would, for the time being, enable backpressure and disallow concurrent reuse due to not being able to (for now) uphold the guarantee required by wasi:http/service where outgoing requests are call-stack-connected to incoming requests.

and C#, because none of them have a precedent for implicit context inheritance

FWIW; .NET does inherit async contexts (ExecutionContext) by default. Both when using regular await syntax and when spawning new tasks. Generally you'd need to go out of your way to opt-out of it (ExecutionContext.SuppressFlow).
System.Diagnostics (tldr: .net's flavor of OTEL) makes use of that to build async-aware traces.

From what I can see, languages and frameworks which support spawning guest-level tasks/coroutines/futures/etc. such that the spawned task has no implicit or explicit connection to the spawning task or its task-local context are the rule rather than the exception. We see this in Rust/Tokio, Go, and C#; and even if the app developer doesn't directly make use of that feature they'll end up using a dependency that does sooner or later.

What I'm hearing now is that any spawned task that might call an import must implicitly or explicitly inherit the spawning task's context -- not just for wasi:http/service but for any WIT world. And a guest runtime which does not yet support that must apply backpressure to prevent any concurrent reuse, regardless of the target world. Is that correct?

That's not what I'm trying to say, no. Instead, I think there is a pattern here that goes beyond the specific CDN-Loop header: the need (or at least desire) to attribute async work back to the event that triggered it. In some cases it's vital for that attribution to be correct, whereas in other cases (like telemetry) it allows the host to provide richer information.

But even setting that aside, I think even just for CDN-Loop, we absolutely shouldn't introduce one version of an outgoing http handler with attribution, which can only be used in exactly one context (sending outgoing requests as part of processing an incoming request that itself was received via a special version of the incoming handler), and another one without the ability to provide attribution.

Nit: just because a guest runtime can make use of implicit, context-derived attribution doesn't mean it should. We can modify the Go scheduler to do this, but we definitely shouldn't, because it assigns meaning to the relationship between the spawning goroutine and the spawned goroutine which is outside the language semantics and unrelated to the intent or expectations of the author, especially given that we're mostly talking about pre-existing library code which didn't have Wasm, the component model, or task attribution in mind at all when it was written.

To the degree that that preexisting code can't use the implicit context without changes, how would it work with the explicit context without changes? And conversely, if we can somehow provide bindings that make the explicit context passing work for preexisting code, why wouldn't that work for the implicit (on the CM level) way of doing it?

Is the distinction you're making between well-behaved and non-well-behaved code something that exists outside of this discussion? I.e. would a JS developer who knows nothing about our newly minted attribution scheme agree about what constitutes misbehavior here? I ask because I'm concerned about compatibility with the larger ecosystem of pre-existing code which may not conform to our idea of good behavior.

What I (unsuccessfully) tried to convey is that the runtime can manage tasks in a way that JS code doesn't need to be changed in any way for correct attribution to fall out from idiomatic use of APIs as they exist in the ecosystem. You'd have to do your own in-JS scheduling and task tracking in ways that'd be highly unidiomatic to get attribution wrong. And if you're doing that, the only thing we could do to enable you to get correct attribution is to make context tracking explicit in a way that I don't think there is any precedent for.

As mentioned, the JS runtime could be implemented in a way where it'd have to do its own context tracking internally, which would allow things like coalescing timers across top-level tasks (and their associated event loops). But that's a runtime implementer's choice, and a design that can fully make use of the implicit tracking isn't particularly more complex or onerous. (I initially thought it was, but thinking through it in more detail made me realize that it isn't really.)

and C#, because none of them have a precedent for implicit context inheritance

FWIW; .NET does inherit async contexts (ExecutionContext) by default. Both when using regular await syntax and when spawning new tasks. Generally you'd need to go out of your way to opt-out of it (ExecutionContext.SuppressFlow). System.Diagnostics (tldr: .net's flavor of OTEL) makes use of that to build async-aware traces.

Thanks; I stand corrected. I was looking at Task.Run and AsyncLocal and didn't see anything about inheritance, so I assumed there wasn't any.

That's not what I'm trying to say, no. Instead, I think there is a pattern here that goes beyond the specific CDN-Loop header: the need (or at least desire) to attribute async work back to the event that triggered it. In some cases it's vital for that attribution to be correct, whereas in other cases (like telemetry) it allows the host to provide richer information.

Sounds reasonable. In that case, if this attribution requirement is neither pervasive nor unusual, and if we're not going to reflect this requirement in the type system, I'd at least like to see some kind of annotation at the WIT and component ABI level to hint to the binding generator that it should add e.g. an extra Context parameter to each annotated export and import for languages where explicit context passing is the norm. Relying exclusively on documentation doesn't help the guest toolchain in its mission to facilitate task annotation where it's needed (and not do so where it's not needed).

To the degree that that preexisting code can't use the implicit context without changes, how would it work with the explicit context without changes? And conversely, if we can somehow provide bindings that make the explicit context passing work for preexisting code, why wouldn't that work for the implicit (on the CM level) way of doing it?

I think the best we can do for languages and frameworks that have an explicit context passing idiom is to generate bindings for each attribution-requiring import and export that has a context parameter. That tells the application programmer that the import can't be called without supplying a context parameter, and thus they should be wary of passing control through a third-party library that might lose track of the context on the way from the export call to the import call.

In contrast, leaving it up to documentation alone would not only make it easier to do the wrong thing but also violate the idiom of explicit context passing, leading the developer to believe they have no responsibility for passing context since they never received a context parameter.

What I (unsuccessfully) tried to convey is that the runtime can manage tasks in a way that JS code doesn't need to be changed in any way for correct attribution to fall out from idiomatic use of APIs as they exist in the ecosystem. You'd have to do your own in-JS scheduling and task tracking in ways that'd be highly unidiomatic to get attribution wrong.

Ok, it sounds like such in-JS scheduling would be considered weird and unidiomatic (and hopefully rare?), at least. I'm mainly trying to establish what "well-behaved" and "not-well-behaved" mean so we can document that for developers. And point out that the definition may differ based on language and/or framework.

I think hints might be an interesting approach, yeah. Though we don't really have a mechanism to support those right now.

Alternatively / additionally, I think we should have a test suite focused on guest language implementations that'd include tests for this kind of thing. That way, guest runtime implementers would get failing tests pointing them in the right direction instead of having to read and hopefully follow documentation

Sounds reasonable. In that case, if this attribution requirement is neither pervasive nor unusual, and if we're not going to reflect this requirement in the type system, I'd at least like to see some kind of annotation at the WIT and component ABI level to hint to the binding generator that it should add e.g. an extra Context parameter to each annotated export and import for languages where explicit context passing is the norm. Relying exclusively on documentation doesn't help the guest toolchain in its mission to facilitate task annotation where it's needed (and not do so where it's not needed).

I agree that capturing this purely in documentation is unsatisfying, and capturing it in the type system somehow would be best. If we release wasip3 without changing the wits to represent this invariant in the CM type system, then we will be under pressure to find a mechanism to add features to the component model type system in a way that keeps the released interfaces forwards-compatible but allows us make the invariant mechanical in future releases.

I expect that adding this annotation in a forwards-compatible way will add considerably more complexity to the CM type system than resolving this as a breaking change to CM-async and/or the wasi wits before wasip3 is released. The cost of that complexity will be borne over a long period, whereas the cost of delaying a wasip3 release to find a better breaking CM and/or wit change is finite and paid now. My current inclination is that we should delay wasip3 until we figure this out, but I want to see if anyone else is aligned before I commit to that position.

My current inclination is that we should delay wasip3 until we figure this out, but I want to see if anyone else is aligned before I commit to that position.

I'll quote what I wrote earlier:

On a practical and process level, the difference is between large-scale changes to WIT interfaces and a purely additive, ABI-level mechanism that doesn't require changing existing interfaces. I think this late in the game, the bar for the former should be "the world is fundamentally broken without this change".

I really really think that the concerns here do not meet that bar, given the huge amount of churn and substantial delays this would incur. If the audience for which "documentation or encoding in the interface" is be relevant would be all application developers, I might see it differently. But given that that audience is guest runtime implementers, and that we can (and should completely independently of this issue) give them a test suite that tests the behavior, what we have now is very far from fundamentally broken.

I think we’re all on the same page here that we want guest runtimes to behave well, and for implementers to have a good time implementing it too. There does however seem to be some disagreement about how to proceed. To summarize, the options we’re currently discussing are:

Type System: Go back to the drawing board on C-M async to design a new (breaking) feature to add support for this directly into the type system. Given the design, implementation, and coordination necessary, the lead time for this would be measured in months. Edit: see wasi-http 0.3: instance reuse means middleware world cannot implement CDN-Loop #919 (comment)
Attribute: Introduce a WIT-level hint for this that guest bindings generators can mechanically pick up on. This is still a change, but no longer breaking, and would exist outside of the type system. This is an easier change and would likely have a lead time measured in weeks.
Documentation: Document an implementation requirement that guest bindings generators need to follow. This would likely be implemented by special-casing the generator based on symbols/paths. This is the easiest change: Document concurrent instance reuse in wasi:http/service #920 encodes it on the WASI side and this is something we can ship with today.

A number of people here have already elaborated why a type system extension at this point would not be the right choice, and I don't have much to add. Instead I’d like to look more closely at the difference between the second and third approaches, because I believe we should be able to change from one to the other in a regular point release.

Example

The way I see this play out is that in 0.3.0 we ship with comments included. This is essentially something like this:

// Handle incoming HTTP requests
//
// # Invariants
//
// Guest bindings generators should create a new context for
// each instance, and connect imports with exports. This is
// necessary to be able to support concurrent instance reuse.
interface handler { .. }

Then in 0.3.x, when we land the support for machine-readable annotations of this kind, we can directly replace that to the same effect:

@scoped // ← encodes the same invariant as the comment
interface handler { .. }

The @scoped notation is just a placeholder notation meant to illustrate the point, not a concrete proposal. The only thing that changes between both examples is how we encode the invariants, not which invariants we encode. That means there shouldn’t be any issues going from one to the other. So the question I want to extend to the group here is: Does anyone see anticipate any forward compatibility issues if we document the requirements in a comment first, and in a future release change that to a first-class machine-interpretable annotation?

Document concurrent instance reuse in `wasi:http/service`

Type System: Go back to the drawing board on C-M async to design a new (breaking) feature to add support for this directly into the type system. Given the design, implementation, and coordination necessary, the lead time for this would be measured in months.

I'll jump back in to point out: this is quite a mischaracterization of the proposal above. The proposal is not to change the type system itself. The suggestion, boiled down, is: "add a parameter to functions". That uses the type system, sure (mistakes will be pointed out by signature mismatches), but it doesn't change it.

(My view is that "add a parameter to some functions" is far less disruptive and uncertain than adding new mechanisms for attributes, which is actually closer to "change the type system". Likewise, a new implicit state slot for current-task, with getters and setters, is closer to "change the underlying system", a larger lift. As everyone else, I want to see things shipped as soon as possible with a durable, simple solution. But the main point of this comment is to point out the mischaracterization above.)

@cfallin

@cfallin Ah yeah, you're right to point out I did misunderstand what your proposal entails. Sorry about that. Admittedly I didn't read that as closely as I should have, and I incorrectly inferred that later mentions in this thread about "changing the type system" referred to your proposal.

@lann

After chatting with various folks about this synchronously, I'm feeling a lot better about this. Here are a couple of insights that were helpful:

@lann pointed out that, for Go, we'll want the generated bindings for each async import or export to include a Context parameter unconditionally in order to support idiomatic cancellation. So my idea about using the presence or absence of that parameter to indicate to the app developer whether they need to link a given import call to an export call won't fly. Which I'm a little sad about, but it makes sense. Consequently, I'm not so concerned about having a WIT annotation/hint to direct bindings generation in wit-bindgen-go.
For me, at least, it helps to frame this as a carrot we're offering to guest runtime developers and app developers, rather than a stick we're threatening them with. We're offering them more accurate CDN-Loop tracking and/or otel tracing in exchange for possibly doing a bit more work (or at least not using "unusual" control flow schemes) to preserve task identity from export call to import call. If they care about those things (and they usually will), we'll give them the tools they need to cooperate in alignment with idioms of the language they're using, and in many cases it will require no extra effort on their part. If they don't care (or don't read the docs), and they end up doing something that breaks the connection from export to import (e.g. use a fresh, empty Context when calling an import in Go rather than threading it through from the export), the component is still valid and the host will run it; they just won't necessarily get the carrot we offered them (e.g. accurate automatic tracing).

Consequently, I'm fine with shipping what we have as-is, using backpressure in the guest to control instance reuse as appropriate, then adding the intrinsic described in WebAssembly/component-model#659 and making it available in guest toolchains, and finally adding a hint annotation to WIT if and when it seems appropriate.

@alexcrichton

My current inclination is that we should delay wasip3 until we figure this out, but I want to see if anyone else is aligned before I commit to that position.

Thanks @alexcrichton @dicej @cfallin @yoshuawuyts and others who we discussed this issue with synchronously, I now have a better understanding of all of the trade-offs for this particular issue and I now agree that the proposals discussed here for documentation now and a forwards-compatible annotation added to the interface later are the appropriate way to deal with this issue. At the conclusion of those discussions, no-one believed there was any reason to delay wasip3.

Document concurrent instance reuse in `wasi:http/service` (WebAssembl…

Navigation Menu

Repository navigation

wasi-http 0.3: instance reuse means middleware world cannot implement CDN-Loop #919

9 remaining items

Example

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

wasi-http 0.3: instance reuse means middleware world cannot implement CDN-Loop #919

Description

Activity

cfallin commented on May 27, 2026

alexcrichton commented on May 28, 2026

alexcrichton commented on May 28, 2026

cfallin commented on May 28, 2026

pchickey commented on May 28, 2026

cfallin commented on May 28, 2026

cfallin commented on May 28, 2026

pchickey commented on May 28, 2026

alexcrichton commented on May 28, 2026

cfallin commented on May 28, 2026

alexcrichton commented on May 29, 2026

lukewagner commented on May 29, 2026

cfallin commented on May 29, 2026

cfallin commented on May 29, 2026

alexcrichton commented on May 29, 2026

9 remaining items

badeend commented on Jun 8, 2026

tschneidereit commented on Jun 8, 2026

dicej commented on Jun 8, 2026

dicej commented on Jun 8, 2026

tschneidereit commented on Jun 9, 2026

pchickey commented on Jun 9, 2026

tschneidereit commented on Jun 9, 2026

yoshuawuyts commented on Jun 10, 2026

Example

cfallin commented on Jun 10, 2026

yoshuawuyts commented on Jun 10, 2026

dicej commented on Jun 10, 2026

pchickey commented on Jun 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions