Non-blocking I/O without cancellation

Hi! The concurrency explainer contains an example on how to implement non-blocking IO.

write() calls stream.write, forwarding the caller's buffer.

If stream.write returns that it successfully copied some bytes without
blocking, write() returns success.

Otherwise, to avoid blocking:

write() calls [stream.cancel-write] to regain ownership of the
caller's buffer.

If select() has not indicated that this file descriptor is ready,
write() starts a zero-length write and returns EWOULDBLOCK.

Otherwise, to avoid the potential infinite loop:

write() copies the contents of the caller's buffer into an
internal buffer, starts a new stream.write to complete in the
background using the internal buffer, and then returns success.

The above logic implicitly waits for this background stream.write
to complete before the file descriptor is considered ready again.

But it also says:

Cancellation is cooperative, (...) allowing the subtask to continue executing for an arbitrary amount of time

Is there a contradiction here?

If the other end of a stream doesn't respect cancellation immediately (which is allowed) or doesn't support cancellation at all (also allowed) the guest will still end up being blocked.

Personally I see this as a consequence of implementing a readiness-based I/O system on a completion-based I/O system. Some compromises will have to be made somewhere, and this is a case where that shows up. If a stream doesn't actually support cancellation then, yes, it'll cause nonblocking I/O to actually be blocking. In practice though I'd expect this to be rare for "expected to work" nonblocking I/O (e.g. TCP/UDP). For file, however, in Wasmtime today cancellation doesn't actually cancel so it means that cancelling a read on a file would actually just block waiting for the I/O to finish anyway.

@badeend

@badeend Great observation! Indeed that was written before async became an effect type and thus the assumption was that a synchronous stream.cancel-write would be used and that that would be fine in practice because, as @alexcrichton mentioned, stream/future read/write cancellation should basically never block b/c, when the other end is wasm, cancellation is spec-defined to complete immediately and when the other end is host, until we have a host doing Overlapped I/O or io_uring, there should be no reason that the host can't cancel the read/write immediately as well. However, that's obviously not a super-satisfying answer.

So one optimization idea that I mentioned a while back (when we first discussed this non-blocking issue) seems like it might also be the fix to avoid this trapping corner case. The idea is to add {stream,future}.try-{read,write} built-ins that work like normal {stream,future}.{read,write} except that, when they returned BLOCKED, instead of continuing the read/write in the background (thereby requiring cancellation), they leave the stream/future in the initial state (with buffer ownership returned to the caller).

I'm not sure when we should do this (maybe as part of a 0.3.x follow-up batch that includes stream.splice), but it seems like a good idea to do before 1.0 so that this is a non-issue.

Is the expectation that the stream.try-{read,write} primitives would both (a) be guaranteed to make progress eventually, if checked repeatedly, and (b) be guaranteed to never suspend? This is a bit difficult for our planned p3 I/O model in hyperlight-wasm, because we are/were planning for a purely completion-based interface: all of the driver state for e.g. a stream provided by the host lives outside a virtual machine boundary, with a relatively slow and more-or-less-inherently async call required to get access to it from the guest. This means that for e.g. the TCP recv case, even finding out whether there are packets available on the NIC/host network stack requires an async/suspending call. We could of course have an extra buffering layer in the wasm runtime when these primitives are used, but I think it would remove the possibility of 0-copy, specifically when these are used.

@syntactically

@syntactically You may also be interested in this thread, which discusses the try- operations in more detail.

@badeend

@badeend Ah, thanks for finding the link, and sorry for forgetting that you were in fact the one who first mentioned this idea!

@syntactically My guess is that stream.try-read would not be expected to make progress on its own and, rather, it would be the async 0-length stream.read that would tell the host, if it's using a completion-based API, to go ahead and read into a host buffer and signal the 0-length read as completed once there is something in the buffer (b/c how else would the 0-length read ever complete and allow the guest to make progress?).

For writes, I think there are two ways it could go, depending on whether the host wants to buffer or not. If the host buffers, then stream.try-write would succeed by writing into a buffer, but fail if the buffer was full which you would then wait on via async 0-length stream.write. If the host does not buffer but is completion-based, I would think that stream.try-write is allowed to always fail and 0-length writes are (already) allowed to always succeed, in which case, according to the rules, must be handled by the guest by buffering the write in guest memory, flushing asynchronously via stream.write.

"Buffering" may not sound great, but if stream.try-{read,write} are being used for fundamentally non-blocking/readiness-based source-code APIs, then I think buffering has to happen somewhere, and if the host is using completion-based APIs, we're effectively just moving this buffering from the kernel to the user-space host or the guest wasm.

You may also be interested in #441 (comment) thread, which discusses the try- operations in more detail.

Thanks! I was looking for that thread but somehow couldn't find it.

If the host does not buffer but is completion-based, I would think that stream.try-write is allowed to always fail and 0-length writes are (already) allowed to always succeed, in which case, according to the rules, must be handled by the guest by buffering the write in guest memory, flushing asynchronously via stream.write.

Ah, that makes more sense. I was expecting this case to be banned if try-write were to be added as its own thing, but if the intention is to keep it then everything's fine :)

rather, it would be the async 0-length stream.read that would tell the host, if it's using a completion-based API, to go ahead and read into a host buffer and signal the 0-length read as completed once there is something in the buffer

From my reading of the concurrency section, the same approach as in the write case (where the guest detects that 0-length reads are not actually signalling readiness and does buffering itself, because try-read always fails after read) would be possible here too?

"Buffering" may not sound great, but if stream.try-{read,write} are being used for fundamentally non-blocking/readiness-based source-code APIs, then I think buffering has to happen somewhere, and if the host is using completion-based APIs, we're effectively just moving this buffering from the kernel to the user-space host or the guest wasm.

I agree with this. I think having it in the guest wasm makes a lot of sense, since that's what can hopefully actually know whether it needs it. I worry with host-side buffering that it might be easy to get into a state where buffering was happening but not needed (e.g. some early-on call did a 0-length write but then later switched to a readiness-based approach on the same fd).

Another related issue is this one. The wasi:sockets UDP interface doesn't use streams, but it runs into essentially the same problem in that: wasi-libc wants to implement non-blocking UDP sends, but with the current idiomatic interface this is generally impossible to do efficiently.
At the time of writing, the consensus in that ticket is leaning toward preemptively copying every buffer. One possible way to avoid this would be to also add "try" variants in WIT here.

It's not directly related to streams, but I'd like to converge the readiness-based I/O story for WASI APIs to whatever the C/M ends up doing.

I'm not sure when we should do this (maybe as part of a 0.3.x follow-up batch that includes stream.splice), but it seems like a good idea to do before 1.0 so that this is a non-issue.

I'm not sure about the timing either. Now that p3 is reaching the stage where implementations are being built, three closely related issues have appeared within a week: 1, 2, and 3.

Maybe this is just recency bias, but it does make it feel like 0.3.0 won't have a clear story for readiness-based libraries right out of the gate. Reading across these threads, the current guidance for guests seems to be either:

issue a blocking cancellation and hope it won't trap, or
defensively copy the buffer, in places where that wasn't necessary in 0.2.x

Some language ecosystems (e.g. Python) are already planning to skip 0.2 entirely because of the churn in WASI. If proper support for readiness-based guests (which realistically covers most existing libraries) is postponed to a later 0.3.x release, that may add another round of churn for early adopters. Streams are a core part of the value proposition for p3, so getting them (ahem) "wrong" the first time could be more harmful than last-minute adjustments. I dislike last-minute changes as much as the next person, but maybe something worth considering.

Anyhow, regardless of timelines:

Various possibilities were discussed earlier. Perhaps we could start with a minimal version:

Add stream.try-{read,write} built-ins.
- Guaranteed never to block.
- No async canonopt required.
- No manual cancellation needed.
If the operation cannot be performed immediately, they return canceled instead of blocked. The buffer is immediately returned to the caller, with no data copied.
Once try-write returns canceled, the guest may not call it again until it has issued (and awaited!) a successful regular non-0-sized write; otherwise, a trap occurs. This ensures forward progress for all combinations of {readiness,completion}-based producer <-> {readiness,completion}-based consumer.

The net result of this protocol:

if either the producer or the consumer is completion-based -> everything is fine.
if both the producer and the consumer are readiness-based -> the C/M places responsibility of providing a buffer onto the producer.

I'm not sure about the timing either. Now that p3 is reaching the stage where implementations are being built, three closely related issues have appeared within a week: WebAssembly/WASI#897, #617, and #618.

Maybe this is just recency bias, but it does make it feel like 0.3.0 won't have a clear story for readiness-based libraries right out of the gate.

It’s painful but I think extremely valuable. Implementing guests and hosts efficiently is what makes wasip3 impactful. I don’t quite know if having all languages go through wasi-libc was the plan all along or not, but with that path it may make modifying the spec less painful than it was before; if languages don’t think about wasip3 themselves and you can rebase on a new wasi-libc.

The net result of this protocol:

if either the producer or the consumer is completion-based -> everything is fine.

if both the producer and the consumer are readiness-based -> the C/M places responsibility of providing a buffer onto the producer.

This summary confused me from the perspective of a host implementing read/writes for host-provided streams, until I realised that it makes sense when looking at streams between two components. The case of a host trying to implement a stream via a zerocopy completion-based I/O API doesn't fit incredibly neatly into this taxonomy, because there's never a time that it's safe to signal readiness if the underlying completion-based I/O API doesn't have some sense of readiness/returning fast enough that it's safe to wait for it, so the guest still needs to buffer and send off a true async (allowed to suspend, and to be slow/impossible to cancel) call (but arguably for a different reason than in the readiness-based consumer case). However, I think the same rule for detecting when buffering is needed still works in this case.

The similar issue in WebAssembly/WASI#897 is interesting to me, because it implies that the same issue of wanting to be readiness-based occurs outside of streams where the 0-byte read/write convention doesn't work. It would be nice not to avoid having multiple conventions for readiness; I wonder if the convention suggested there, of having try-write return a future that indicates readiness would also work for try-{read,write} on streams.

It feels like this is getting at something quite fundamental. I wonder if there will be more use cases for this "readiness-style" of async outside of primitive I/O.

@badeend

@badeend Personally, I buy the reasoning for adding try variants in 0.3.0. If we added try as an optional immediate (with a new opcode in the binary format to minimize churn) as opposed to new built-in, that might be both less of a pain, and I think semantically try ends up being a boolean parameter to the existing read/write paths as opposed to something wholly new, so altogether it might not be a huge lift to include in 0.3.0, but I'd be interested to hear what others think.

My impression is that WASI#897, while also about "cancellation", is a distinct issue (for reasons I just mentioned in that issue) and that there's no component-model fix suggested there (other than perhaps to make streams more expressive in some additive, later 0.3.x release so that UDP could use streams just like TCP). But it's definitely good to juxtapose these issue and ask the question of whether there's some single underlying issue.

@lukewagner

In talking about this some more with @lukewagner and @dicej today one thing I wanted to write down here as well is that personally I don't think we can avoid cancellations entirely. With the goal of giving affordance to completion-based systems it seems that no matter what variants of try-* are added or what wasi-libc does there will always eventually be a case of wasi-libc having open file descriptors with in-progress I/O operations on them. For example today it's required when 0-length reads/writes don't work, and with try-* it's required to determine readiness when the try-* blocks. In this situation close(fd) is fundamentally a non-asynchronous function and somehow needs to be handled. That leaves two possible implementations:

close performs a blocking cancel
close deallocates what it can but allows the full closure to proceed in the background (somehow)

With (1) although try-* might make it so libc read and write functions don't ever use a blocking cancel it'd still be in libc via close. With (2) I'd personally be worried about getting resources into weird states where close cleans up the libc-based fd but doesn't actually clean up the underlying host resource. For example files would remain open, ports would remain bound, etc. This means that subsequent operations aren't guaranteed that the previous resources are actually released, which I've seen is often a constraint when using close for example (e.g. not being able to delete an open file on Windows, ports not being able to be re-bound for the next test, etc).

Personally I feel that (2) here is hazardous enough that I'd like to avoid it. Thus assuming libc were instead to take on (1) as an approach, issuing blocking cancellations in close, then it's still not actually possible to do non-blocking I/O without cancellations. Additions of try-* functions would mean that read and write don't have to do a blocking cancellation, yes, but it would mean that a theoretical high-level goal of never needing blocking cancellation would remain unfulfilled.

From a concrete perspective we could consider the impact of using wasi-libc in a synchronous component model function context. In such a context it's not possible to call blocking functions, but it is possible to call async functions (e.g. issue I/O ops). As-is today it would mean that read and write libc functions (nonblocking) could not be used with a sync component model function because of their issuance of blocking cancellation. That would change with try-* where read and write would be valid, but close would continue to be invalid. However we could also change wasi-libc to use an async cancel + block on cancel instead of a blocking cancel. This would mean that guests would only dynamically trap if cancellation blocked and wouldn't trap if dynamically cancellation all worked out.

Overall, to me there's a spectrum of possibilities here:

Do nothing - read, write, and close cannot be used reliably in sync function contexts.
Use async cancel - read, write, and close would work "most of the time" in a sync function context, but this is not a hard guarantee
Add try-* functions - read and write would be guaranteed to work in a sync function context, but close would not.
Add try-* and use async cancel - this requires effectively a background thread to perform the "real cleanup" and means that host/OS resources are kept alive beyond a call to close if necessary.

Overall, to me, none of these are particularly satisfying. I don't find that any of these necessarily neatly solves the problems at hand here. I agree with @syntactically that this touches on the fundamental nature of completion-vs-readiness-based I/O, and I basically think there's only so much that can be done with what we have.

W.r.t. timelines -- I agree we want 0.3.0 to be solid. I think it's best to first determine what the end state we want is, and then we can work backwards from there how exactly to ship and implement things.

So, overall, I'm left with -- what's the fundamental problem we're trying to solve? Is it to remove all blocking calls to cancellation in libc? Is it to only remove cancellation in read and write? Is it to remove blocking cancellation from the component model entirely? Basically I feel like I need more clarity on this part to better evaluate/discuss possible options here.

@alexcrichton

I think the primary goal should be to make c/python/ruby/rust on top of wasi-libc behave as expected and for language maintainers to be happy relying on wasi-libc. I think that means select/O_NONBLOCK guest code today does not introduce new pauses, buffers, or requires stack switching. But perhaps that is too much; maybe making some blocking cancellation calls is okay; maybe relying on sync lowering is okay also.

A host that uses epoll behind the scenes and promises instant cancellations of read/connect/etc. will behave like libc-on-linux today with no API changes, but that introduces hidden assumptions and I don't think another component could implement that. It seems to me now that either you sacrifice guest behavior (by requiring completion IO); host freedom of implementation (by requiring readiness IO); or API complexity (by providing both readiness and completion IO).

I do not have a good sense for what language teams might want and/or would be happy accepting behavior-wise. My perhaps uniformed opinion that completion IO is still rare, the performance benefits are small for wasm today, and if performance were to become a problem in the future an extra API would be an acceptable cost then. At the same time, I don't know how to implement readiness IO on the component model today in a clean way without having try- methods everywhere; and then the read(0) like @alexcrichton mentions would have to not block closing, and, well, ugh.

what's the fundamental problem we're trying to solve?

My take:

Support all combinations of {readiness,completion}-based producers <-> {readiness,completion}-based consumers for both hosts and guests
Non-blocking APIs should reliably not block
Avoid redundant memory copies
Keep cancellation cooperative and async, for all the reasons outlined in (A)synchronous cancellation guarantees #618

Essentially, I want to have my cake and eat it too, but I actually think this is all achievable if we're willing to tolerate that close() indeed can block (as it already does).

Even outside WASI, blocking during close() is standard:

Sockets with SO_LINGER: The socket(7) man page states that close() won't return until queued messages are sent or the linger timeout expires
Filesystem descriptors: Linux's "close-to-open cache consistency" for NFS causes close() to block until pending changes are written, ensuring subsequent open() calls see the latest data
The POSIX docs mention two other cases, but those are not directly relevant to current-day WASI.

I think its OK and inevitable for close() to block in wasi-libc; it does so in POSIX too. Though people tend to blissfully ignore this corner case. There isn't much they can do about it either ¯\(ツ)/¯

close performs a blocking cancel

Actually, close() should wait on in progress I/O rather than cancel it. After a successful write/send(), callers should be able to assume it is safe to close() without data loss.

close deallocates what it can but allows the full closure to proceed in the background (somehow)
(..) Personally I feel that (2) here is hazardous enough that I'd like to avoid it

Agree. If guests want to prevent traps from a blocking close(), they should handle it themselves on a background task (somehow)

Alex' third option is exactly what I had in mind:

Add try-* functions - read and write would be guaranteed to work in a sync function context, but close would not.

Is it to remove blocking cancellation from the component model entirely?

No, quite the contrary. I think the removal of blocking cancellation was mixed in with this discussion because it is currently the only way to emulate non-blocking IO. If the C/M provides native support for readiness-based operations through e.g. try options, I don't see async cancellation as a concern.

Is it to remove all blocking calls to cancellation in libc?

Not a standalone goal. However, with try options in place, the need for cancellation in libc naturally disappears.

Is it to only remove cancellation in read and write?

Those are the most critical cases. The goal is to remove blocking from any API that claims to be non-blocking.

Overall, to me there's a spectrum of possibilities here:

Do nothing - read, write, and close cannot be used reliably in sync function contexts.

Use async cancel - read, write, and close would work "most of the time" in a sync function context, but this is not a hard guarantee

Add try-* functions - read and write would be guaranteed to work in a sync function context, but close would not.

Add try-* and use async cancel - this requires effectively a background thread to perform the "real cleanup" and means that host/OS resources are kept alive beyond a call to close if necessary.

One thing I'm especially concerned about here is the risk of components assuming too much about the runtime. I think I'm quite against the 2nd option here, because as far as I'm understanding, it means that the trapping behavior of the component (not even just its performance behaviour) is dependent on specific host implementation choices. That would mean that we could have a perfectly spec-compliant completion-based host in hyperlight-wasm, but find that a lot of peoples' default-tooling-generated components worked perfectly on wasmtime on linux which they might use for local development, but trapped in hyperlight-wasm when actually deployed. (I imagine this would also be a concern for other cloud runtimes). I would want to try to avoid that as much as possible.

what's the fundamental problem we're trying to solve?

My take:

Support all combinations of {readiness,completion}-based producers <-> {readiness,completion}-based consumers for both hosts and guests

To clarify: Am I understanding correctly that this is only a goal for specific, limited categories of "primitive" I/O that we expect language standard libraries to themselves expect to be nonblocking? Specifically, actual read/write on streams (e.g. from files, and maybe sockets) seems to be all we are discussing here. So, the readiness side of this is more about "make sure that the bindings that come out of the bindings generator are, for an interface written to expect it, usable by a standard library that expects nonblocking/readiness-based IO" than a much more general "I need a bindings generator to be able to create a readiness story for any arbitrary async import that I might have"?

I think its OK and inevitable for close() to block in wasi-libc; it does so in POSIX too. Though people tend to blissfully ignore this corner case. There isn't much they can do about it either ¯(ツ)/¯

One difference is that AFAIUI the blocking will only cause performance problems (on a rare operation) in most POSIX/native software, whereas #578 means that there are cases in the component model where blocking in close will cause traps that basically destroy the entire world, rather than just cause performance pathologies. I would expect that to mean that it is a good deal harder to ignore. I don't have a good sense from the language side of the degree to which that might be a problem, though.

@jellevandenhooff

This is a bit of a long comment, but the tl;dr; is that personally I feel like try-* intrinsics should be a strong consideration or 0.3.x, but not required out-the-gate. What follows is a bunch of rationale and discussion...

One motivation I have here is that I think we need to make a decision ASAP about whether a change needs to happen now or in an 0.3.x release. To make such a decision, though, I want to explore the various consequences, decisions, etc, of what exactly we would release and what are the consequences of not releasing it.

So in light of that, I want to concretely explore the idea of adding {stream,future}.try-{read,write} intrinsics. First, the consequences of NOT having these intrinsics are:

From a component-model perspective, async I/O is not async. Synchronous functions in the component model cannot call wasi-libc read and write robustly. This is a dynamic trap that would come out when any opportunistic operation doesn't complete immediately, whether or not the "other half" of the stream/future is a guest or a host.
Hosts are the only possible way to actually induce blocking for streams/futures. For async func either guests or hosts can block in their cancellation. Practically there are no known guest bindings that would block in cancellation (Rust e.g. just drops the future). Host bindings for streams/futures are known to block for things like filesystems. Folks generally don't expose POSIX nonblocking I/O to work on files though.
It's probably not the most performant to issue an I/O operation, then cancel it, then issue a zero-sized I/O operation.

The benefits of having these intrinsics is that it more-or-less solves all of the above points. The only one it doesn't touch on is the async func properties (which is relevant for UDP). For UDP though the try-* functions could be modeled in WIT most likely.

However, I also want to highlight properties of async I/O both with and without try-* intrinsics:

Blocking cancellation is not going away. This concept still exists at the component model level and no component model runtime would be complete without supporting it.
Blocking cancellation (or blocking of some form) is still going to exist in close. This means that close is never 100% reliable to call from a synchronous component model function.
Internal libc-based buffering is always going to be required. In a situation where the host is using completion-based I/O, and the guest is using POSIX nonblocking I/O, the decision tree for write is -- (a) try-write returns it was cancelled, (b) zero-length writes are determined to not work (they're "always ready"), so (c) write must buffer data internally in wasi-libc, return success from write, and have the write go into the "background" (and fully resolved on close)
The state machine for zero-sized in-flight writes either working or not working is fundamentally required in wasi-libc.

Given all that, I want to explore some consequences/respond to various comments.

I think the primary goal should be to make c/python/ruby/rust on top of wasi-libc behave as expected and for language maintainers to be happy relying on wasi-libc.

I would agree with this as well @jellevandenhooff, yeah. This goal gives us quite a bit of leverage and leeway as well, too, because we can centralize work in wasi-libc. For this goal, for example, I would consider it acceptable to do try-* intrinsics in a subsequent 0.3.x release for the component model since it would impact the performance/usability of guests, but now how they're built. In the meantime all major entrypoints into components are expected to be async-typed, so it would give a lot of runway before the more niche situations where try-* shines to show up, and thus consequently more time for us to implement it.

I think that means select/O_NONBLOCK guest code today does not introduce new pauses, buffers, or requires stack switching.

Personally my thinking here is that if the only consequence for guests is traps in sync component model functions for now, but a restriction lifted later, I would consider this acceptable to defer to an 0.3.x release. In libc select and close will fundamentally do blocking things, but I think that matches what libc users would expect. For read and write the blocking, if any, would largely be transparent and would be considered a bug to fix, and likely one that wouldn't show up for awhile.

I think its OK and inevitable for close() to block in wasi-libc; it does so in POSIX too.

...

Actually, close() should wait on in progress I/O rather than cancel it. After a successful write/send(), callers should be able to assume it is safe to close() without data loss.

Oh that's an excellent find and an excellent point @badeend, thanks for that! That's good to know that there's at least precedent in libc, although I would share the same concerns in that if in practice close is never blocking then it would mean we can't really lawyer our way into "well technically your code was alreay broken". I think this gives us license to block in close, which would be a consequence for libc-based guests running on a purely completion-based host, but in a sense it'll likely still be surprising when it happens.

Is it to remove all blocking calls to cancellation in libc?

Not a standalone goal. However, with try options in place, the need for cancellation in libc naturally disappears.

To clarify @badeend, the need for cancellation in the read and write functions disappears, but the need in general is still present (e.g. close), right?

Is it to only remove cancellation in read and write?

Those are the most critical cases. The goal is to remove blocking from any API that claims to be non-blocking.

@badeend do you have a better sense/feeling for the consequences of delaying try-* to 0.3.x? My hunch is that while it would be nice to read and write in a synchronous function context in the component model it's also somewhat niche and going to take some time to crop up. Effectively I'd be willing to be that we could implement try-* and update wasi-libc before anyone actually hits surprising behavior in practice, but I'm curious if you feel the same way.

but find that a lot of peoples' default-tooling-generated components worked perfectly on wasmtime on linux which they might use for local development, but trapped in hyperlight-wasm when actually deployed

@syntactically question for you on this sort of -- the trapping behavior here only happens during synchronous function types in the component model. My expectation is the main poster-child for I/O in a synchronous function is printf, but otherwise I wouldn't actually expect it to show up that often.

Put another way: technically wasi-libc will need to maintain a sort of allow-list of functions that can be called in a synchronous function context. Today that does not include read and write, and tomorrow it would with try-*, but at the same time I don't know what actual consequences this would have. In my experience most folks tend to be blisfully unaware (rightfully so) of how exactly their application interacts with libc because otherwise it requires complete and total knowledge of the entire system to make any sort of judgement. What I'm getting at is that I think there's no escaping the concern you have here. Even if read and write were fixed or changed, just dropping anything anywhere in Rust would issue a close and all of a sudden could mean a synchronous function traps.

Effectively I think this is going to be a fundamental property no matter what, so I'm not sure this should have bearing on try-* personally, but I'm curious how you feel.

To clarify: Am I understanding correctly that this is only a goal for specific, limited categories of "primitive" I/O that we expect language standard libraries to themselves expect to be nonblocking?

At least for me @syntactically I've mostly been concerned with wasi-libc around this issue in the face of component model interfaces. (which is to say, either a completion or readiness based host) In that sense it's little to do with bindings since whatever APIs we create for components we'll attempt to expose their full expressiveness natively in languages (e.g. FutureReader and StreamReader in Rust). So no a bindings generation concern, but how specifically wasi-libc (and other POSIX applications) should operate in the face of a completion-based API.

In that sense "works on linux but traps in hyperlight" would be a wasi-libc bug IMO, but something relatively easily fixable and something we can centralize in libc, in theory. This is separate though from dynamic runtime behavior, e.g. the "do async cancel then block on the result" which would work in Wasmtime but fail in hyperlight-wasm -- but this is more of a question of APIs used rather than bindings or any particular flavor of I/O.

I'm starting to think my performance arguments cuts both ways; if I claim io_uring performance doesn't matter because wasi has plenty of overhead anyway, then perhaps having guests do an extra copy if they want to model readiness based IO internally is fine. So then perhaps taking the component model as-is, declaring p3 done, and seeing how the cards play out with wasi-libc and adoption isn't so bad. I am sure there are plenty of other optimizations to be made, and no one really knows where the issues are until there is adoption. It would be so much nicer to ground arguments in concrete experiments with performance numbers, and those experiments will be so much easier to run if we have end-to-end languages running on real runtimes.

I don't know how open everyone is to future changes after wasip3 if things turn out difficult. With the wasi-libc approach I think making changes would be less painful than if every language had their own deeply integrated bindings... but it would still be hairy. (And if this is a can of worms not to open here then that makes also.)

@badeend

Based on @badeend's observation that there is indeed precedent for close() to block and wait for pending I/O to complete, then it does seem like there is a principled goal state where O_NONBLOCKing read()/write() never block while select()/close() do, allowing code aiming for maximal concurrency or running in a synchronous context to behave accordingly.

(On the topic of "behave accordingly", a question for @badeend: if one performs a graceful non-blocking shutdown on a socket (iiuc: shutdown() followed by select()ing for POLLIN and then read()ing EOF?), I suppose that would also guarantee close() was non-blocking, since there'd never be a pending stream.write to wait for?)

That all being said, I expect @alexcrichton is right that we have some time before anyone hits this in practice and since the solution we all seem to be agreeing on (try variants for stream.{read,write}) is additive and non-breaking, it seems like we could do it in a 0.3.x minor release.

@badeend

Based on @badeend's observation that there is indeed precedent for close() to block and wait for pending I/O to complete, then it does seem like there is a principled goal state where O_NONBLOCKing read()/write() never block while select()/close() do, allowing code aiming for maximal concurrency or running in a synchronous context to behave accordingly.

Would that work in browsers though? I'm not sure what argument we could make for accepting deviations from run-to-completion semantics for these operations. Otherwise I agree that this makes sense

@tschneidereit

@tschneidereit If close() blocks, it'll trap if called from a non-async export, which, in a browser context, means that Jco can use JSPI to wait for any Promise used to implement any pending stream.write if need be. Practically speaking, we're probably in an async context if we're doing libc file I/O (how else do we select()), so hopefully this shouldn't bite folks in practice (nor the temporary lack of try variants used for read()/write(), assuming that printf()/stdio are implemented with synchronous console calls and never actually block).

Ah yeah, that makes sense: I see how close isn't a concern for studio nearly as much as for more general io. It'll certainly cause weird issues for people in some corner cases, but seems like the least-bad of the options we actually have

a question for @badeend: if one performs a graceful non-blocking shutdown on a socket (iiuc: shutdown() followed by select()ing for POLLIN and then read()ing EOF?), I suppose that would also guarantee close() was non-blocking, since there'd never be a pending stream.write to wait for?)

shutdown on TCP sockets is always non-blocking, but doesn't hook into the poll mechanism and is not awaitable from user space. The kernel finishes the connection in the background, potentially even after the process has already ended. close does not wait for this. Unless SO_LINGER is enabled; in that case close blocks until the connection is fully closed (or the specified timeout expired).

You could interpret this as: by default the kernel takes the route of option 2 from above:

close deallocates what it can but allows the full closure to proceed in the background (somehow)

and SO_LINGER inverts this behavior by pulling the blocking finalization into the "foregound"; blocking userspace (but again: not awaitable by poll)

After thinking about it some more, I'm coming around to the idea of pushing it to a later 0.3.x release.

If a readiness-based guest wants to be absolutely sure that I/O never blocks; then they should not follow the current guidelines and instead preemptively copy the data into an intermediate buffer before issuing the read/write. Taking that as a starting point, the proposed try option is "only" a performance optimization and doesn't add any expressivity nor does it (un)block any use cases.

Would that work in browsers though?

I would keep in mind, technically ECMA-262 ("JavaScript") doesn't have I/O at all. So, any presumptive guarantees WebAssembly makes with regard to I/O, and JavaScript is involved, requires an asterisk.

If a readiness-based guest wants to be absolutely sure that I/O never blocks;

Tricky.

E.g., if you test 5 or 6, or 10 or 12 JavaScript engines, runtimes, or interpreters you WILL find that some might have I/O, some might not. Last time I checked StarlingMonkey, js-compute-runtime, CloudFlare Workerd do not have the capability to read STDIN and write to STDOUT.

A JavaScript implementation can have no I/O at all, e.g., SerenityOS's LibJS do not have the capability to read STDIN or write to STDOUT, and beconformant with ECMA-262.

Further, if you test 10 or 12 JavaScript runtimes reading STDIN and writing to STDOUT, some might buffer 16384 (Bun), some might read 1 MiB in one read (QuickJS). Some might be able to import C into the engine and compile JavaScript Ahead-Of-Time to a standalone executable, and internally be using C's STDIN or STDOUT, and process I/O just as fast as C, C++, Rust. Some, such as V8's d8 simple can't read STDIN in binary form in a meaningful way. Some, such as Mozilla's SpiderMonkey might require some attention to detail such as always sending \n at the end of a write for the engine to be able to read the input.

So, I'd be very careful about what you specify and presumptively guarantee with regard to reading and writing I/O. Test - using at least 20 different programming languages and runtimes ("guests"; "hosts") before you write something out that winds up being empirically provable as inaccurate or even impossible in the field.

Block on pending writes in `close` (#845)

Navigation Menu

Repository navigation

Non-blocking I/O without cancellation #617

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Non-blocking I/O without cancellation #617

Description

Activity

alexcrichton commented on Mar 5, 2026

lukewagner commented on Mar 6, 2026

syntactically commented on Mar 11, 2026

badeend commented on Mar 11, 2026

lukewagner commented on Mar 11, 2026

syntactically commented on Mar 11, 2026

badeend commented on Mar 11, 2026

jellevandenhooff commented on Mar 11, 2026

syntactically commented on Mar 12, 2026

lukewagner commented on Mar 12, 2026

alexcrichton commented on Mar 12, 2026

jellevandenhooff commented on Mar 13, 2026

badeend commented on Mar 14, 2026

syntactically commented on Mar 16, 2026

alexcrichton commented on Mar 16, 2026

jellevandenhooff commented on Mar 18, 2026

lukewagner commented on Mar 23, 2026

tschneidereit commented on Mar 23, 2026

lukewagner commented on Mar 23, 2026

tschneidereit commented on Mar 23, 2026

badeend commented on Mar 23, 2026

guest271314 commented on Apr 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions