Suggestion: Combine opaque external references with explicit wasm-extern casts

In issue #293, we discussed the possibility that funcref not be a subtype of anyref, to avoid representation-change costs at the JS/Wasm boundary. It is reasonable to assume that other types might also require similar transformations in the future: i31ref will most probably require some form of range check, and it is not unlikely that structs and arrays will require some wrapping to function optimally in both worlds.
This creates a situation where passing a value to wasm as anyref carries significant cost at the boundary, since every object has to be checked against every Wasm type, in case a representation change has to be applied. The idea to apply these checks during downcasts from anyref will not work, as we would need to apply the reverse transformation when upcasting to anyref, breaking the fundamental assumption that upcasts are no-ops.
Therefore I suggest the following (assuming funcref <: anyref, but this does not matter for this discussion):

Reintroduce an externref type which represents host references in the host's representation. This will be separate from the type hierarchy, i.e. not a subtype of anyref.
Specify that anyref is not just a supertype of funcref, i31ref, and dataref, but in fact contains no values that do not classify as one of these three types. In other words, anyref is the type of all references that can be generated by a wasm module.
Introduce explicit casts from anyref to externref and vice versa. These will not be free, and will perform the same representation change as at the Wasm/JS boundary. For performance, we will also introduce additional casts from externref to more specific types. The specifics of the representation change will of course depend on the module's host.

With this design

an API that wants to pass opaque references to Wasm can pass them as externref and have the guarantee that there is no cost at the boundary. Such references might even be Wasm references (albeit in the host's representation); this way, another Wasm module can create objects internally and pass them to an externref interface.
an API that wants to pass references of specific Wasm types can do so, accepting the cost of representation change. Note that this is likely to be quite minimal for more concrete types than anyref.

I understand that the premise of this post is not a given. However, I also do not see the downside: even if we decide that funcref </: anyref, we allow for more flexibility in the implementation of other (current and future) reference types, and allow Wasm modules to specify more cleanly whether they expose an opaque interface.

What I like about this idea is that it's a general solution that addresses several issues that came up in previous discussions (such as i31-vs-host-tagging-schemes, costs created by funcref <: anyref) by separating concerns and making certain costs pay-as-you-go (instead of imposing them on every case). Thereby it lets us define the core Wasm type hierarchy however we see fit without worrying too much about costs incurred when anyref is used at the module boundary. And I think it avoids all the pitfalls that plagued earlier suggestions (including my own) around possible relationships between anyref, externref, "opaqueref" etc.

@rossberg

+1 to reintroducing externref; I think we removed it prematurely.

From what I glean above, the anyref would in fact be a wasmref and the two types are not related, thus requiring explicit casts both directions. Did I get that right?

One concern that @rossberg has brought up in that context is a Wasm module should be virtualizable as a host module and vice versa. We'd need to be able to specify a type above these two, or have type imports, for that to work.

Perhaps a different approach is to have type imports. An externref could then be a type import provided by the host.

@rossberg

From what I glean above, the anyref would in fact be a wasmref and the two types are not related, thus requiring explicit casts both directions. Did I get that right?

Correct. These would be the same "casts" that are happening in the boundary.

One concern that @rossberg has brought up in that context is a Wasm module should be virtualizable as a host module and vice versa. We'd need to be able to specify a type above these two, or have type imports, for that to work.

Would that not be achieved by creating references in the host, then casting them to externref (if needed) and passing them to the hosted module?

Correct. These would be the same "casts" that are happening in the boundary.

I agree that these casts are essentially hidden in the JS embedding, so making them explicit is attractive. That does have a downside in that they impose a cost on cross-module wasm -> wasm calls that use externref to communicate.

I also like the idea of externref </: anyref. E.g., this would allow SM to, if it wanted, represent externref with its existing 64-bit NaN-boxed JS value representation while representing anyref with a SMI/pointer word. So, along with funcref </: anyref, this would give us 3 "root" supertypes with 3 distinct representations which seems pragmatic.

Incidentally, I don't think we can emulate externref with type imports: the idea with type imports is that you import a subtype of some fixed supertype. Thus, you could only type-import a subtype of any, func, data or any other heaptype. This "can only import subtypes of a fixed heaptype" restriction is key in allowing us to compile type imports as a fixed blob of machine code without monomorphization (using the "universal representation" of the import's supertype).

That does have a downside in that they impose a cost on cross-module wasm -> wasm calls that use externref to communicate.

Right, but the downside compared the the current state would exist only in the case where they communicate directly though externref, and these values never cross the boundary. Modules could expose an anyref and an externref API if this starts becoming measurable.

IIUC (please correct me if I'm wrong), in the presence of funcref <: anyref, one way to phrase the suggested hierarchy is

      may have casting cost
               |
     anyref   <->   externref
     /    \
dataref  funcref
   |
  ...

where anyref is kinda a typedref that can perhaps one day be obtained directly through type imports. Yet, this doesn't seem to solve the representation problem on the anyref/funcref edge - or does it?

If it does not, a likely (here: consolidated) alternative hierarchy with funcref </: anyref could become

      may have casting cost or may fail
             |               |
  dataref   <->   funcref   <->   externref
     |
    ...

yet this one seems odd on the background that a) from a producer's point of view (regardless of representation issues), the most viable/convenient/expected hierarchy would be the initially envisioned

     anyref
      /  \
dataref  funcref
   |
  ...

and b) given the former dataref/funcref/externref separation, if there is a JS import that accepts any JS/Wasm value (and would hence prefer to use the original anyref), the import needs to be duplicated twice to accept either a dataref, funcref or externref, offloading the representation change to the Wasm<->JS stub, unless (some of) these conversions are permitted implicitly to avoid the duplication, but doing either of these leads back to square one at the boundary anyway it seems.

What am I missing? :)

Casts from dataref or funcref to externref should always succeed, thus the import could accept externref.

@lukewagner

A few observations:

We have identified 3 possible universes of representation:

(1) external: references in host-prescribed representation
(2) internal: references in native Wasm representation
(3) function: function references in Wasm representation
Internal includes both i31 and data refs – these must be in the same universe and have a shared representation and a common supertype, otherwise core use cases of i31ref would be broken. (But as long as extern is separate, there is no problem with that.)
All external references ought to be convertible into internal references somehow (at least at the boundary), otherwise virtualisation would be broken in one direction. Furthermore, externref, if not fully internalisable, would always enforce extra boxing, even on platforms where it wouldn't be needed. Consequently, we need to allow that internal references can be formed from all external references, including those that were formed from functions (which does not imply that they are castable to functions or have the same representation).

Putting this together, there is the following conceptual hierarchy:

            ___________ any? _________          
           /             |            \
     extern            intern          func
    /      \          /  |   \
(host)  (exported) i31  data  (imported)
        /   |    \             /      \
 (e_i31) (e_data) (e_func)  (i_host) (i_func)

where the parenthesised nodes are types that we may not necessarily name in Wasm. Here, exported are the native Wasm values converted/wrapped to be represented at the host side, and dually, imported are host values converted/wrapped to be represented uniformly with native Wasm values (this may include wrapped Wasm functions).

If any does not exist, then explicit conversions will be required between these types (these are conversions, as opposed to casts, since they can involve representation changes):

extern.convert_intern : [internref] -> [externref]
intern.convert_extern : [externref] -> [internref]
extern.convert_func : [funcref] -> [externref]
func.convert_extern : [externref] -> [funcref]  ;; may trap

and for completeness, maybe also

intern.convert_func : [funcref] -> [internref]
func.convert_intern : [internref] -> [funcref]  ;; may trap

If any exists, then the diagram collapses imported = extern - func and exported = intern + func, necessarily (and e_* = * and i_* = *).

But if any does not exist (at least to unify intern and extern), then every module will have to pick a preferred representation between intern and extern. We need to be aware that this reduces composability. The same problem is created for type imports, as @lukewagner points out.

That is, every module will have to choose whether it is optimising for linking against the host or linking against another Wasm module. Note that this does not just affect exports, but internal choices like type definitions for data structures, so having two versions for every export function does not solve the problem (nor would it scale). Furthermore, the problem extends to exports like mutable tables and globals, which you cannot duplicate anyway.

So, while I think separating the types is possible, there is no free lunch, and this certainly creates its own set of complexities and problems (some of which may be difficult to predict).

@rossberg

At the last subgroup meeting we all agreed that figuring out the type hierarchy was a high priority, so it would be good to move this discussion forward as much as we can.

I was thinking about the top any type in @rossberg's diagram above and wondering whether we could get the benefit of having externref as a "lazy" internref even if we had that single top type. I think we can, but it would require representation changes on downcasts. Consider a smi that flows from JS into Wasm where an extern reference is expected. Since extern is supposed to be lazy, the engine does not check what kind of reference it is at this point. Later, the value flows to a location with type any and is then downcast to i31. Since it is a smi, the cast succeeds without representation changes. However, if a boxed number followed the same path, then this downcast would require a representation change to i31. (The end result in both cases needs to be the same since a smi and boxed number should not be distinguishable.)

In the OP, @manoskouk wrote:

The idea to apply these checks during downcasts from anyref will not work, as we would need to apply the reverse transformation when upcasting to anyref, breaking the fundamental assumption that upcasts are no-ops.

But I don't see why we would need to apply reverse transformations on upcasts to any. In the example above the i31 you get after the downcast can flow anywhere an any is expected with no further modifications. Is there another case where upcasts require representation changes that I'm missing?

Also, are representation changes on downcasts acceptable? I could imagine this could get hairy if the engine moved code around so that the value to be cast was materialized in multiple locations (e.g. in two different registers). Then the cast would have to update the representation in multiple locations.

The only case that I can think of a representation on downcast being a problem is if both the from and to type are both under eqref. IIUC the intention is that ref.eq is always a word compare.

As for multiple places to update representations, a cast produces a new value and leaves the old value unchanged.

Is there another case where upcasts require representation changes that I'm missing?

Other types may have different representations in JS and Wasm. In V8, these are currently functions, but it may expand in the future.
This means that if an upcast from funcref to anyref does not do the reverse transformation, you end up in a spot where an anyref value may have either representation. This complicates further downcasts, as well as crossing the boundary to JS.

I very much like the framing of this as making boundary conversions optional and explicit. With this in mind, it makes sense to extend the conversions to any Wasm type, i.e. not just reference types. We can convert to/from any Wasm type at the boundary, and the explicit conversions would mirror that capability.

We would have two instructions (names not meant as suggestions):

to_extern: any type -> externref
from_extern *type*: externref -> type

This positions the feature as independent of the funcref <: anyref question. It also takes serious the observation (CMIIW) that, even for reference types, the engine can often do a better job of converting a value from externref to a known final type than converting to internref and then downcasting.

In this model, we might want to rename the externref type to reflect that it's not necessarily a reference type in the same sense as the other reference types (i.e. it's not part of the hierarchy and is not composed of a heap type and a nullability).

Pondering more over the additional instructions that we'll need to support the 3 separate hierarchies, I realised the following:

Because Wasm functions can be exported to JS, they will have a representation as extern values.
Because all extern values can be converted to intern values, Wasm functions also have a representation as intern values!
Consequently, functions do not have 2 but 3 different conceptual representations in our design! In particular, there are two competing representations inside Wasm itself.
Because Wasm code can compose func->extern and extern->intern conversions, and likewise the opposite direction, the third representation can easily be constructed inside Wasm. Conceptually, this is the func subtype of intern that we wanted to remove!
However, we don't name this subtype, nor do we make it usable directly. Instead, we require converting it back to func first. But we could equally well introduce proper structural types for it and have a version of call_ref operating on it directly.
Which raises the question why we bother with the separate func representation when the one under intern is unavoidable anyway. Effectively, we don't get rid of it, we merely make it artificially unusable, and introduce a redundant representation instead.

Maybe it's too late over here, but at this point, the design we chose looks increasingly silly, and I wonder if we should reconsider the decision to have a separate representation for functions in the MVP. Note that we can also add it later, as the separate type of "raw" code pointers that it is.

Thoughts?

Good points, and I don't think they've been raised before. I don't think it would entirely violate the spirit of our consensus to consider changing the three-pronged approach to a two-pronged approach, since fundamentally the question we answered was whether we should move forward with a single top type.

That being said, I don't think these points invalidate the reasons we had for keeping func separate, namely that we would like to keep the door open for engines to have a different representation for func. In an engine that has a different func representation, both the intern and extern representations of functions would be boxed in some way, so the explicit conversion back to func before calling would make a hidden conversion cost explicit.

Unless anyone feels strongly that these new considerations change the picture, we should stay the course with the three-pronged design.

Could an even more minimal option be to not attempt to provide any coercions between funcref and anything else? This would allow funcref to be its own separate type, just like i64 (which can also be exported to JS and would also have the same issue if we attempted to, e.g., provide i64-to-externref coercions).

Apologies for missing the meeting, I have a recurring conflict at exactly that time and have to juggle.

I think we should go with the three-pronged consensus for now, in the interest of having a narrow enough cone of uncertainty to iron out the binary format and have tests for it. I love the part where I have some code to write and some tests that tell me if that code is working. Let's get to that part.

There still seems to be disagreement on where the burden of proof lies w.r.t to the any design question, though. We should move forward now, but if we want to revisit the decision, then what new information or data is required?

@rossberg

@rossberg: I think I see your point, but I don't think this creates a silly situation: conceptually, I think of Wasm funcrefs exported to JS as "JS wrapper objects that (internally) reference a Wasm function" (whether they're actually implemented as wrapper objects is irrelevant). So if such a JS object is passed to Wasm as an externref, then it's just another opaque host object. Converting it to internref doesn't change its representation, and doesn't make it usable in any way other than as an opaque pointer -- not because we're explicitly barring "the third kind of funcref" from being useful, but because opaque host objects simply aren't usable in general. In that sense, I agree with @lukewagner 's i64 example, which applies to all numeric types: e.g. a WebAssembly f64 can be exported to JS, where it'll be a JS Number, which can be passed back to Wasm as externref, but that doesn't really mean that "there's a silly unusable f64 representation that's a subtype of anyref", it's just another opaque host object.

@lukewagner

@lukewagner, since func refs can already escape into the JS universe, and the conversion happens implicitly at the boundary, we cannot truly remove that conversion. But you are right that we can probably make it unobservable from within Wasm, analogously to number types, as you suggested. That makes sense to me.

Okay, another question.

In the past, we repeatedly talked about the possibility of Wasm reference types that are not allowed to be passed to JS (similar to v128, e.g., consider continuation objects from the stack switching proposal). What would this mean for the intern->extern conversion lowered into Wasm?

A non-option is to make the conversion trap or otherwise fail. That is not viable, because whether something is allowed to escape to the host is fundamentally a policy decision that may differ between hosts, and we don't want the Wasm conversion semantics depend on the host. (As an example, we may not want to allow continuation values to escape to JS, while it would be undesirable to make them inaccessible through the C API.)
Then the only other option is that there are checks at the boundary, in environments that restrict the values. That would be fine by me, but it implies that the use of externref is still not check-free: in a host with such restrictions, it will require a case analysis at the boundary, but this time on the way out. Are we okay with that?
Edit: a third option is to never reject any references at the boundary either. Instead, just make them sufficiently inaccessible in JS. But I don't know if that's enough to cover all potential cases. In particular, that would still allow them to roundtrip back into Wasm in potentially undesirable ways. I guess we lack a good picture of the possible scenarios.

In the past, we repeatedly talked about the possibility of Wasm reference types that are not allowed to be passed to JS (similar to v128, e.g., consider continuation objects from the stack switching proposal). What would this mean for the intern->extern conversion lowered into Wasm?

Could such values be lossily converted at the intern->extern step to a completely opaque "sentinel/invalid" externref that can't be downcast/converted back to the original Wasm value? This would capture that such values have no real lossless extern representation while keeping the overhead of handling this confined to the conversion points.

@conrad-watt

@conrad-watt, that would fall under the "otherwise fail" clause I mentioned. It still means that the semantics is host-dependent. In particular, you can't know whether intern->extern followed by extern->intern succeeds and returns the original value. So, no.

@rossberg

@rossberg I was under the impression is that extern serves the role of representing "already legal" external host values. If some intern references are not legal as extern, depending on the host, it would seem like the intern -> extern explicit conversions are in fact, host-dependent. Maybe that shouldn't be an instruction then, but an imported function from the host?

I think the "never fail" option makes a lot of sense. Just like externval lets us box all foreign host values so they can roundtrip host→wasm→host, it seems reasonable for the host to provide the inverse box, allowing all wasm values to roundtrip wasm→host→wasm. In a JS setting, I imagine this taking the form of some JS API-defined exotic object that stores the wasm value in an internal slot but provides no way to get at it, so the only thing you can do with it is pass it back to the JS API. I could even imagine this being a quite useful feature for linker/loader-type JS glue code scenarios.

@lukewagner

@lukewagner, I think I agree. Though one reason to prevent values from crossing is not to expose identity/equality to JS, when they are not eqrefs in Wasm, e.g., to avoid exposing implementation details. OTOH, we already have that issue with funcrefs, and they are exposed.

@titzer, I agree that host-dependent behaviour would be fine if it was an import. But I'm not sure that we have a good model yet for such a "system import", or that it would meet the expectations of the group.

@rossberg

@rossberg Yeah, that's a good concern. I wonder if maybe JS exotic object semantics give us hooks for making object identity unobservable? Otherwise, it seems like we could specify a "new object every time" semantics which would probably be fine.

As per today's meeting, I think we don't need a wasm type, since we've split the hierarchy into the three prongs.

I'll go ahead and close this issue. Any follow-up discussions can be new issues.

Navigation Menu

Repository navigation

Suggestion: Combine opaque external references with explicit wasm-extern casts #307

60 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Suggestion: Combine opaque external references with explicit wasm-extern casts #307

Description

Activity

jakobkummerow commented on Jun 10, 2022

titzer commented on Jun 10, 2022

fgmccabe commented on Jun 10, 2022

manoskouk commented on Jun 10, 2022

titzer commented on Jun 10, 2022

lukewagner commented on Jun 10, 2022

manoskouk commented on Jun 11, 2022

dcodeIO commented on Jun 11, 2022

manoskouk commented on Jun 11, 2022

rossberg commented on Jun 13, 2022

tlively commented on Jun 23, 2022

titzer commented on Jun 24, 2022

manoskouk commented on Jun 24, 2022

askeksa-google commented on Jun 24, 2022

60 remaining items

rossberg commented on Jul 12, 2022

tlively commented on Jul 12, 2022

lukewagner commented on Jul 12, 2022

titzer commented on Jul 13, 2022

jakobkummerow commented on Jul 13, 2022

rossberg commented on Jul 13, 2022

conrad-watt commented on Jul 13, 2022

rossberg commented on Jul 13, 2022

titzer commented on Jul 13, 2022

lukewagner commented on Jul 13, 2022

rossberg commented on Jul 14, 2022

lukewagner commented on Jul 15, 2022

titzer commented on Jul 26, 2022

tlively commented on Jul 26, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions