Repository navigation
Weak imports #36
Description
Wasm doesn't have a "weak" import mechanism, which would allow one to (import "foo" "bar") where "bar" may or may not exist in "foo" (or perhaps where "foo" itself may not exist). This would be one way to allow feature detection in APIs.
As one possible solution, it's expected that the wasm GC proposal will eventually give wasm the concept of optional/nullable references, which would permit a kind of weak import where one imports global variables with reference type which can be either references to the desired entities or absent/null. However, GC may not be available in wasm for some time, and even when it is, it may not be implemented in all wasm engines, so it's not entirely obvious whether we should depend on it for something so fundamental to WASI.
Another is to essentially add a weak-import system on top of Wasm. For example, we could use a naming convention, such as a ".weak" suffix, to indicate a weak import, and a ".is_present" suffix on a global variable import to hold a boolean indicating whether a weak import was successfully resolved, like this:
(func $foo (import "wasi_something" "foo.weak"))
(global $do_we_have_foo (import "wasi_something" "foo.is_present") i32)A similar scheme could enable detection of modules too:
(func $foo (import "wasi_something.weak" "foo"))
(global $do_we_have_wasi_something (import "wasi_something.is_present" "foo") i32)or even both at the same time:
(func $qux (import "wasi_something.weak" "bar.weak"))
(global $do_we_have_it (import "wasi_something.is_present" "foo.is_present") i32)Another option would be to have a dynamic API that takes a string, however while we can do that, passing around strings introduces some complexity and it seems like if have a naming convention for ".weak", we might as well have ".is_present" too.
I'm not attached to any of the details here, I'm mainly just looking to start a discussion.
Activity
programmerjake commentedon May 11, 2019
I would assume that calling
wasi_something.weakwhen it is not present would result in a trap.programmerjake commentedon May 11, 2019
One solution is to support nullable references only for importing variables/functions as a subset of the GC proposal. That seems trivial enough that I would expect all WASM implementations to be able to easily implement that, since it doesn't actually require GC.
devsnek commentedon May 11, 2019
this definitely seems more like a general wasm problem than a wasi problem. i do like what programmerjake said.
dumblob commentedon May 11, 2019
I'm definitely raising my hand for not making WASI dependent on a garbage collector. GC is meant to be fully optional and an efficient implementation will be a significant code base compared to the rest of WASM runtime.
In that sense, I'm supporting what @programmerjake proposed as it's exactly as @devsnek says - not a WASI problem, but rather a WASM problem.
Btw. I find any naming convention as wrong solution when proposed from the beginning as to me naming conventions try to solve rather some problems discovered later - and having two such conventions already in the standard makes a really bad impression. In my opinion weak imports shall be first class citizens in WASI/WASM.
rossberg commentedon May 11, 2019
FWIW, there has been a general sentiment that the typed function references proposal, which brings typed references into Wasm, should also include nullable ones. While I didn't have time yet to update the proposal, that's the current plan. So nullable refs are to be split off from the GC proposal already.
Whether they are the best way to provide feature detection is a different question. Dynamic feature detection is a dead end anyway, since you cannot opt into types that way. That, however, would be necessary for APIs that introduce new host types (e.g., imported reference types). So in general, you'll need link-time feature detection.
sbc100 commentedon May 12, 2019
I can think of two other possible solutions:
Switch to a an interface query model where a WASI module only imports a single "query_interface" function which can retrieve the set of function pointers (table entries) for a given WASI module. All the types for the module would be known statically but the function addresses would be retrieved dynamically.
Functions could be imported statically but as addresses (table entries) rather than functions:
(global $foo_addr (import "wasi_somthing" "foo_addr") i32)These could then be compared with 0 (or some other fixes, invalid, table entry).
rossberg commentedon May 13, 2019
The general case I have in mind is an API that defines its own host reference types. Consider a file system module for example, which might provide a
filetype. If you want to avoid using anyref for everything (and thereby dynamic type checks in the API), then this type has to be imported into Wasm, to be even able to express the types of corresponding functions, whether on imports or on call_indirect statements. There is no way in which you can make that import dynamic.(Also, I'd argue that the implication of requiring call_indirect for every API call is defeating half the purpose of the Wasm module system.)
The problem with dynamic feature checking in general is that it conflates program execution with program configuration. For scalable modular software it's better to keep those separate -- at least that's what all larger software systems do. Feature testing is a hack that was invented in languages that did not have any proper support for modularity or versioning at the time and where programs remained small enough to keep that tractable.
titzer commentedon May 13, 2019
If I understand Dan's proposal correctly, the ".weak" and ".is_resolved" suffixes would require no changes to wasm's import mechanism, it is a just a convention in WASI. (Presumably, as @programmerjake suggested, trying to call a non-resolved weak import would trap). That seems relatively clean, and preferable to crossing into the dynamic world of nullable function references. Those have some overhead, both in metadata and possibly runtime, whereas imports have less overhead since they are only bound once at instantiation. And I agree with @rossberg that import calls via references and indirection seem like the wrong workaround.
sbc100 commentedon May 13, 2019
Surely calling
.is_presentor.is_resolvedis runtime still feature detection?It does avoid the call_indirect and allow direct imports I agree is preferable. On the downside it does double the number of imports and require the embedder to create fake trapping functions for each weakly undefined functions (one per signature I guess?)
sunfishcode commentedon May 13, 2019
Yes, I meant to say above that an unresolved ".weak" function would trap.
And yes, ".is_present" enables dynamic feature detection. This is useful. Requiring programs to be split into modules for each API feature they need to detect would be a considerable burden, even on programs written in languages with modern module systems.
if (global.get $foo.is_present)is dramatically simpler than putting code in a separate module, attempting instantiation, and catching the link failure.I even wonder whether we could extend the ".weak" runtime detection to work for types too. Rough idea: Allow ".weak" on type import names, and then have any imported function with a signature that mentions an unresolved weak-import type trap. Unresolved weak-import types would otherwise act like
anyref(after validation). It seems like that should be compatible with wasm's existing import mechanism as well.PoignardAzur commentedon May 17, 2019
I don't really understand the interest of a specialized weak import mechanism, given the possibility of a more general optional type for imports.
Would .weak and .is_present have use cases that optref wouldn't cover?
sbc100 commentedon May 17, 2019
My understanding is that this proposal is designed to work in lieu of a first class weak import in the core spec and that we could/should move over to spec'd weak imports once such a thing exists.
Its also a goal to avoid the extra level of indirection involved with
call_indirectsince these imports are essentially static. Its not clear to me that usingcall.refwill have a similar indirection overhead tocall_indirector not, but it seems likely.binji commentedon May 17, 2019
call.refon an optref should be faster than acall_indirect.call_indirecthas to perform atable.get(with bounds check), null check, type comparison, and then indirect jump.call.refwould only have to check whether the reference is null, then indirect jump.The weak import mechanism could be slightly faster since you can codegen a stub that traps instead of performing a null check, and you can use a direct jump.
14 remaining items
rossberg commentedon Jul 11, 2019
Index spaces are an internal naming mechanism, and as such, an implementation detail of a module that is not observable externally. A custom section mechanism using them would require special access to module internals and would not be polyfillable.
pchickey commentedon Jul 11, 2019
Got it, thanks @rossberg. I am happiest with option 4 then - I'd prefer to put names in the custom section than defining some way to mangle names.
guybedford commentedon Jul 11, 2019
I'm not sure I follow this logic - what is the primary friction seen to be? For example implementing this in Node.js would be trivial.
Add a writeup of current weak imports design
Add a writeup of current weak imports design
Provide DIRCOOKIE_START, FD_STDIN, FD_STDOUT, and FD_STDERR. (WebAsse…
Use a `stream-status` type instead of `bool` for end-of-stream. (WebA…
Add a writeup of current weak imports design
Make `utc-offset` signed. (WebAssembly#36)
Add a writeup of current weak imports design
Merge pull request WebAssembly#36 from rvolosatovs/update/wit-deps
Add a writeup of current weak imports design
Merge pull request WebAssembly#36 from WebAssembly/pch/require_semico…
Merge pull request WebAssembly#36 from Mossaka/new-wits