Limitations of start function with exported memory

We've been experimenting with using the start section in the C/C++/clang/lld world:
https://reviews.llvm.org/D40559

I've run into an issue with exported memory and static initializes (or any code in the start function). Basically no JS function that requires access to memory can be run during start because the exported memory is not available until after the start function completes.

We can switch to importing memory but that doesn't seem like a reasonable requirement.

Am I missing something?

Reading the linked review, it isn't clear to me what you're trying to execute in start. Is it everything before main, but excluding main, or up to and including main? FWIW I don't think start should call main, or that was the original idea for start anyways.

Regardless, it seems like segments can do all of the C static global initializers just fine, but the non-trivial C++ global initializers can't generally be done because they might execute arbitrary user code including syscalls. That's indeed troublesome!

Do you think the problem is only embedder calls trying to use an exported memory, or do you think these calls might also try to re-enter WebAssembly? I think it's a simple enough thing to do if the fix is to export memory before start, then export everything else after. If we instead export everything before start then I'm not sure...

At the same time, start is meant to model what ES6 module's "evaluate" does. In a world where WebAssembly can participate in ES6 modules, what should "evaluate" / start do w.r.t. exports?

@jfbastien

What do you mean by "is not available" during start? Presumably the Memory that is created for import could (in principle) be accessed via a closure while start is executing. Or, is there some annoying check that means you can't read/write to the Memory until it's attached to a fully-instantiated Instance?

I agree that it makes syscall implementation awkward - I thought of this problem too while I was trying to implement some syscalls for upstream Musl. I wasn't too bothered by the Memory issue, but the problem of imports seems thornier - until the Instance is created (at the end of start) there really isn't any way for imported functions to call back into Wasm. In practice, it's OK, but it could cause problems for someone.

The ideal thing would have been that "this" refers to the partially-instantiated Instance if imports are invoked during start, so that an imported function can call exports during start via "this.exports.blah". But that opens up a whole can of safety issues, while the current approach is safe albeit a bit limiting.

For syscalls, it shouldn't matter much. By definition, they are in "kernel-mode" and thus don't need to call anything in "user-mode" (Wasm). Most of the syscalls ought to be able to their thing without touching the Module/Instance.

@jfbastien - in the LLVM review, there's no particular opinion on what runs during start. Currently LLD's default entrypoint is _start (ie run main as well). I agree that's not ideal, I'm hoping that we could change the default to something else like _start_wasm to run everything before main. It's up to whoever runs the compiler which entrypoint you set, and regarding Sam's issue here I don't think it matters whether main is run or not.

@sbc100

@sbc100 Yes, that does seem problematic in the near term. ES Modules do allow cyclic imports and so, if we had ES Module integration, the JS called by wasm could be an ES module that imported the calling wasm module to get the Memory.

@jfbastien

From my POV the problem would be solved by giving the embedder access to the newly created memory before start is called. @jfbastien its not clear to me how you would propose doing this. Can you explain?

In terms of what we will use the start section for, the jury is still out on that. What I've been looking at is moving the C/C++ static init function there, and leaving main to be run explicitly by the embedder. But this issue exists whatever we choose to run there.

@jfbastien

From my POV the problem would be solved by giving the embedder access to the newly created memory before start is called. @jfbastien its not clear to me how you would propose doing this. Can you explain?

I think that what we end up doing should match what ES6 modules do: first they link, then evaluate. This can be circular! In our implementation we use the ES6 module machinery, we first link here, resolving exports and setting the corresponding symbol table. After all the ES6 modules are linked (there's only one for WebAssembly at the moment) there's a call to evaluate which sets up element and segment, and at the very end we call the start function. That's the same as running an ES6 module's global code just before returning to the import call (and this is done in an order that's determined by link IIUC).

Given this, what I think needs to happen is that we make export availability match what ES6 modules do for their own exports during evaluate. I'd consult with @domenic or @Constellation about this.

I think this also happens to fix what you're trying to do.

But from the JS API POV, how could we allow the developer to get a handle to the new memory before start is called? I was under the impression that the instance would need to be returned in order to get the memory handle, but that the start function is guaranteed to be called before then.

As a workaround I could switch all the test cases to import their memory.. which is probably not a bad idea, but obviously others could still run into this issue.

I'm a bit confused. If you need two-phase initialisation where some init function has to be invoked after the instance has been fully constructed and handed out, then what keeps you from using an ordinary function for that? Or asking the other way round, how would the semantics of a start function whose execution you'd have to initiate explicitly be different from just invoking a normal function?

@rossberg

@rossberg using a normal function invocation is the alternative option and its what we currently do. Its just that that means our tooling conventions (and clang/lld generated code) won't be able to take advantage of the wasm start section. I don't really feel strongly about using the wasm start section, but I just wanted to raise the issue here in case people see it as problematic that C/C++ won't be using it.

This came up again in the llmv bug tracker recently: https://bugs.llvm.org/show_bug.cgi?id=37198

This is coming up again with WASI: WebAssembly/WASI#19

this isn't just a start function problem. if you declare a function in js or wasm, and wasm imports it (esm), that function has no way to get access to the memory of that wasm module.

This issue relates to the case when the wasm memory is exported from the module. In the case when the wasm memory is imported into the module then is seems reasonable to assume that the embedder has access to the memory since it would have needed it in order to instantiate the module in the first place.

There is third case where the memory is neither imported nor exported but of course there is no way to share the memory in that case.

However I agree this issue should probably be renamed since it relates not just to memory but to anything that wasm module might want to export.

In retrospect we should have named the start function "init function", to indicate that its purpose is module initialisation, i.e., that the module isn't intended to be accessible before that initialisation is complete. Allowing reentrancy during initialisation would bring about the well-known issues with exposing uninitialised state, and thereby breaking encapsulation guarantees (which is an important feature of Wasm's module system).

That said, I believe that all discussed cases can be handled by having the module export an explicit init function. The start function isn't for these cases, but is that a problem?

The only problem I see is that often seems to confuse people.

I don't feel strongly about it but perhaps since the name is part of the binary format, we can still rename it to something else like "init" in the text format and in the spec text?

@littledan

@littledan What do you think of the above proposed addition to ESM-integration? This might also be a good in-person CG discussion topic during the ESM-integration slot.

@sokra

Apologies for my delay.

The above idea sounds good to me: we have both the current start function, called before exports are visible, and another function which is called after exports are exposed. Seems like this would meet all of the goals described above. I like how this change is purely local, not requiring a second traversal over the module graph (in addition to not requiring binary rewriting).

Bikeshedding: We could call this *start*, which is quite invalid as an ES identifier, while being typeable on many keyboards easily.

I will include this in my presentation to the Wasm F2F and get a spec PR out ASAP. Shout-out to current implementers in Rollup, Webpack and Node.js that this change is under consideration: @sokra @guybedford @MylesBorins (I intend to follow up with details in your repositories once we make a decision).

In the Ethereum use case (ewasm) we have the same problem and our way of solving this is having an exported function named main. We'd be very much in favour of following a standard agreed here and not inventing our own version.

Would it be possible extending the StartSection to include two start functions:

pre-finalisation (aka pre-exports are loaded) – this is the current "start" function
post-finalisation – this would be the "new start" function

Studing this problem a bit more, I can see that start functions are more complex than what I assumed in #1160 (comment), since:

There may need to be some embedder-specific logic to run before calling the secondary start function, e.g., in this proposed Node.js/WASI integration https://github.com/nodejs/node/pull/27850/files#diff-ab9c666b48467c7030bd93aac6d06eb2R184
Any one particular name that we choose might overlap with something in some other environment, so best not to hard-code one

From here, I think we need to do a more careful study of how various toolchains solve this problem, rather than assuming that a simple solution would as proposed above would work. If you could give pointers to your system's module initialization path, or ideally, summarize more details, in WebAssembly/esm-integration#32, it would be really helpful.

This continues to confuse folks: https://bugs.llvm.org/show_bug.cgi?id=42713

@littledan

(Returning to this because I think it would be quite valuable to standardize in wasm so as to avoid the current ad hoc embedding conventions.)

One way to generalize the Node.js issue @littledan pointed out is: what should the JS API do? If the point of this start_ (better name TBD) function is to be called immediately after exports are visible and WebAssembly.instantiate() or new WebAssembly.Instance() simply calls start_ right after start and before returning the new instance to the caller, the exports are not in fact yet visible.

One idea is that these instantiation functions can take a callback that is passed the just-created exports object and called before the start_ function. So, looking at the Node.js example, you could write:

new WebAssembly.Instance(compiled, reflect.imports, { expose(exports) {
  for (const expt of Object.keys(exports))
    reflect.exports[expt].set(exports[expt]);

  if (usesWasi && exports.memory)
    wasi.setMemory(exports.memory);
});

Thinking about this from an engine impl pov, this expose callback is actually symmetric to how we'd implement ESM-integration: the engine's wasm::Module::instantiate routine would take a native callback that was called between start and start_ and the ESM loader would use this callback to make the exports live in their respective ESM bindings. So really, we'd just be exposing this ESM-integration callback to JS when the lower-level JS API was used.

@littledan @binji WDYT?

i suggested this callback idea somewhere ~~(i honestly have no idea where, it was a while ago) and reasons were presented that it was suboptiomal. i'm trying to find it but i haven't yet.~~ (found it, WebAssembly/WASI#82, on the subject of memory init and resizing, ran into issues with threading, apparently not as relevant as i remembered :P)

more recently, i was under the impression that the plan to fix this was to support passing live slices of memory (fs_read(fd, buf) instead of fs_read(fd, u32IndexToBuf)), or whatever other things are possible to be imported/exported from wasm

Great question: passing live slices would indeed avoid the need for the wasi.setMemory(). And native wasm ESM-integration would remove the rest of it, I expect. But I think the general problem of "I'm using the JS API to instantiate but I need to do something to make the exports "live" (for whatever my definition of "live" is) before the _start function is called" remains.

I like this line of investigation, and it seems like that could solve this particular issue with WASI but I don't really feel confident answering it in the air. I think, to proceed to make this a standard rather than a WASI-specific loader, it'd be best if we could do a survey of Wasm toolchains to figure out their initialization patterns, and see if this would solve their problems. I don't have time to do this sort of survey right now, personally.

The start function may accept an argument: what to be started. Then the function name will remain. The default argument would be as the specification is defined now: lets say "START_DEFAULT" = "INIT+COMPLETE". Then if you want to break the start to few stages, have "START_INIT", "START_COMPLETE" and so on called separately.

Else I don't see any limitation of the start function as it is now, when its idea is to only initialize the basic functionality of the module for itself. Then, however, practically there is no function that will 'run' the module after its initialization is fully complete in a standard way. One run/main function could be added to facilitate that.

Add a toolchain-independent ABI document, and propose `_initialize`

Add a toolchain-independent ABI document, and propose `_initialize` (#…

Navigation Menu

Repository navigation

Limitations of start function with exported memory #1160

32 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Limitations of start function with exported memory #1160

Description

Activity

jfbastien commented on Nov 30, 2017

NWilson commented on Nov 30, 2017

lukewagner commented on Nov 30, 2017

sbc100 commented on Nov 30, 2017

jfbastien commented on Nov 30, 2017

sbc100 commented on Nov 30, 2017

rossberg commented on Dec 5, 2017

sbc100 commented on Dec 5, 2017

sbc100 commented on May 1, 2018

sbc100 commented on May 17, 2019

devsnek commented on May 18, 2019

sbc100 commented on May 20, 2019

sbc100 commented on May 20, 2019

rossberg commented on May 20, 2019

sbc100 commented on May 20, 2019

32 remaining items

lukewagner commented on May 29, 2019

littledan commented on Jun 11, 2019

axic commented on Jun 12, 2019

littledan commented on Jun 21, 2019

sbc100 commented on Aug 7, 2019

lukewagner commented on Mar 5, 2020

devsnek commented on Mar 5, 2020

lukewagner commented on Mar 5, 2020

littledan commented on Apr 27, 2020

verbessern commented on Apr 27, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions