Repository navigation
Adding an interface to query readiness to accept requests #784
Description
This would serve a few purposes:
-
As an instance reuse "handshake" between the caller and handler, allowing e.g. a host to drop an instance that won't accept any further requests or a guest to eagerly perform some expensive state reset.
-
As a form of flow control (back pressure) for entire requests. This is useful in load-balancer-like situations where the sender wants to pick from only handler(s) that are ready to process a request.
-
A way for handlers to explicitly signal concurrency support.
A rough draft:
interface readiness {
/// May be called to query the handler's readiness to accept request(s).
/// - Blocks if the handler is not ready to accept any more requests but may become ready in the future.
/// - Returns `ok(n)` when the handler is ready to accept up to `n` concurrent requests.
/// - Returns `err` if this instance won't ever accept another request (instead of exiting).
/// - Returns `ok(0)` if the handler cannot report its readiness; the caller will just have to take
/// its chances on `handle` blocking (maybe better as an error variant?).
ready: async func() -> result<u32>;
}This would be added to the proxy world or the proposed service world. Alternatively this function could be added to the handler interface, though I'm not sure that it is applicable to generic "outbound http" imports.
Activity
tschneidereit commentedon Sep 18, 2025
I've thought about this a bunch and talked about it with @dicej and others. So far, my conclusion has been that we already have what we need:
I think the right way for a guest to signal that is to
exit(0).Is your thinking that this would make it so a guest is forced to make a statement about concurrency support, compared to backpressure signaling? Because apart from that it seems like backpressure signaling does what we need here, no?
I think the bar for introducing this should be pretty high at this point, because ideally we wouldn't introduce new things to p3 interfaces anymore.
lann commentedon Sep 18, 2025
This is stemming from a conversation with @dicej and others. 🙂
I think there was a concern that this is not composable, i.e. just because some nested component is done doesn't mean the entire composition must be done.
My understanding is that backpressure is only visible once an export is actually called, which would force the caller to "commit" to making the request (or copy/buffer it). I'm also unsure if the caller can distinguish between a call blocking due to backpressure vs any other reason.
tschneidereit commentedon Sep 19, 2025
I was already sad I couldn't make it to that meeting, and now I'm even more so :/
Hmm. To the degree that really is a problem, how would this proposal address it? It seems like it'd still require the top-level component (or, if that's not a thing as such, whatever component exports the proposed function) to make the decision. It might consult other composed components in doing so, but it could do that before calling
exit(0), too.Obviously the fact that any nested component can call
exit(0)at any time is a composability issue, but that seems to be neither here nor there.That used to be the case, until the introduction of backpressure.{inc,dec}.
lann commentedon Sep 19, 2025
Another approach which would capture some of this functionality would be to wrap the
handlefunction in a resource (similar to this "endpoints" idea) and use the async function returning that resource to signal readiness, e.g.get-endpoint: async func(name: string) -> result<endpoint>, or even astream<endpoint>.My main concern is that if this mechanism is anywhere other than the proxy world (or whatever world ends up best representing "middleware") then its usefulness will be capped by whatever portion of middleware implementations choose to opt-in to it.
[-][v0.3] Adding a function to `handler` to query readiness to accept requests[/-][+][v0.3] Adding an interface to `proxy` to query readiness to accept requests[/+][-][v0.3] Adding an interface to `proxy` to query readiness to accept requests[/-][+][v0.3] Adding an interface to query readiness to accept requests[/+][-][v0.3] Adding an interface to query readiness to accept requests[/-][+]Adding an interface to query readiness to accept requests[/+]support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve`
support WASIp3 instance reuse in `wasmtime serve` (#11807)
support WASIp3 instance reuse in `wasmtime serve` (#11807)
support instance reuse for WASIp3 HTTP components
support instance reuse for WASIp3 HTTP components
support instance reuse for WASIp3 HTTP components