Can WebAssembly.Instance recompile an already-compiled Module?

Say I have this code:

let descriptor = /* ... */;
let memories = Array.apply(null, {length: 1024}).map(() => new WebAssembly.Memory(descriptor));
let instance = fetch('foo.wasm')
  .then(response => response.arrayBuffer())
  .then(buffer => WebAssembly.compile(buffer))
  .then(module => new WebAssembly.Instance(module, { memory: memories[1023] }));

Is WebAssembly.Instance allowed to block for a substantial amount of time. Could it for example recompile the WebAssembly.Module?

In most cases I'd say no, but what if the already-compiled code doesn't particularly like the memory it receives? Say, because that memory is a slow-mode memory and the code was compiled assuming fast-mode? maybe memories[0] was a fast-mode memory, but memories[1023] sure won't be.

What about this code instead:

let instances = [0,1,2,3,4,5,6,7].map(v => fetch(`foo${v}.wasm`)
  .then(response => response.arrayBuffer())
  .then(buffer => WebAssembly.compile(buffer))
  .then(module => new WebAssembly.Instance(module)));

Are those calls to WebAssembly.Instance allowed to cause recompilation?

Assuming the above makes sense, here are a few related questions:

Do we want a promise-returning async function which can compile and instantiate? I'm not saying that we should drop any of the synchronous and asynchronous APIs that we already have, I'm proposing a new asynchronous API.
How does a browser expose that compiled code in a WebAssembly.Module is fast, and that a WebAssembly.Memory instance is suitable for such fast code? Right now the answer seems to be "try it and see if you can notice".
How does a user know how many WebAssembly.Memory instances they're allowed before they get slow code (counting the implicit ones, e.g. as created by the second example)?

Would be nice for WebAssembly.Instance to sometimes cause recompilation, this way immutable globals vars could be constant folded in the generated code. For instance, Emscripten generates relocatable code by offsetting all pointers to static data. The offset is passed in as an immutable global var when the module is instantiated. If WebAssembly.Instance can recompile, it could specialize the generated code.

The specification doesn't define what "compilation" is, nor would it make
sense for it to do so, because implementation approaches may wildly differ
(including interpreters). So it cannot have any normative say about this
either way. The best we could do is adding a note that
WebAssembly.Instance is expected to be "fast".

On 27 October 2016 at 03:24, Michael Bebenita notifications@github.com
wrote:

Would be nice for WebAssembly.Instance to sometimes cause recompilation,
this way immutable globals vars could be constant folded in the generated
code. For instance, Emscripten generates relocatable code by offsetting all
pointers to static data. The offset is passed in as an immutable global var
when the module is instantiated. If WebAssembly.Instance can recompile,
it could specialize the generated code.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#838 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEDOO9sJPgujK3k0f6P7laYV_zaJxES5ks5q3_1LgaJpZM4Kh1gM
.

Agreed this would be a non-normative note at most.

In SM, we are also currently intending for instantiation to never recompile so that there is a predictable compilation cost model for devs (in particular, so that devs can use WebAssembly.compile and IDB to control when they take the compilation hit). Instantiation-time recompilation from within the synchronous Instance constructor would certainly break that cost model and could lead to major jank.

But I do appreciate that separate compilation is fundamentally at odds with a variety of optimizations one might want to do to specialize generated code to ambient parameters. Fusing compilation and instantiation into one async op makes sense and is something we've considered in the past. The downside, of course, is that this inhibits explicit caching (there is no Module), so the developer has to make an unpleasant tradeoff. Some options:

The impl could do implicit content-addressed caching (which could include ambient parameters in the key), like we do with asm.js currently in FF. This would be kindof a pain and has all the predictability/heuristic problems of any implicit cache.
We could create a new way (e.g., a new WebAssembly.Cache API where you pass in bytecode and instantiation parameters and get back a Promise<Instance>.

The latter intrigues me and could provide a much nicer developer experience than using IDB and perhaps a chance to even further optimize caching (since the cache is specialized to purpose), but it's certainly a big feature and something we'd want to take some time to consider.

@lukewagner

@rossberg-chromium I seem to have explained my purpose badly: I don't want to quibble over what the spec says. I'm trying to point out what seems like a serious surprise for developers, hiding under the API. A developer won't expect .compile's result to be re-compiled. That seems like a design flaw to me.

@lukewagner even with implicit or explicit caching we may have the same issue: how many WebAssembly.Memory can be created in the same address-space / origin is a browser limitation. I like what you're suggesting, but I think it's orthogonal to the issue. Let me know if I've misunderstood what you suggest.

Maybe .compile and Module could be given a Memory, and Instance has a .memory property which can be passed to other compilations / instantiations?

I'm not trying to eliminate the possibility of re-compilation, I think we rather want a common idiomatic API usage which has perfect information w.r.t. Memory at first-compile-time (or at cache-retrieval time) so that the compilation emits bounds checks or not if needed.

@jfbastien

@jfbastien With implicit/explicit caching that was provided the particular instantiation parameters (so Memory), I don't see how there would be any need for recompilation.

@jfbastien

@jfbastien With implicit/explicit caching that was provided the particular instantiation parameters (so Memory), I don't see how there would be any need for recompilation.

There may be:

Create many Memorys.
Compile code, with explicit (slow) bounds check because there were too many Memoryies.
Cache that code.
Leave page.
Load page again.
Allocate only one Memory, which gets the fast version.
Get from the cache.
Receive slow code Instance.

At this point I agree you don't need recompilation, but we're being a bit silly of we do slow bounds checks when we don't need to.

As I said: I like this Cache API you propose, I think it makes WebAssembly more usable, but I think the problem is still there. 😢

Well that's my point about having an enhanced cache that accepts instantiation parameters and bytecode: the cache is free to recompile if what it has cached doesn't match the instantiation parameters. So the steps would just be:

create many Memorys
request an Instance from the cache, passing one of those (slow) Memorys
slow-code is compiled, cached and returned as an Instance
leave page
load page again
allocate only one Memory
request an Instance from the cache, passing the fast Memory
fast-code is compiled, cached and returned as an Instance

and after step 8, all future page loads will get cached fast or slow code.

@lukewagner

@lukewagner First of all, you're proposing a mitigation that flies in the face of the stated goal of WebAssembly providing deterministic performance. The difference between slow and fast was last quoted as around 20%, so it would really stink if a spec that painstakingly aims aims for deterministic perf drops it on the floor because of an API quirk. I don't buy that the browser having a content-addressed cache is the right solution, because the spec already goes to a lot of trouble elsewhere to obviate the need for profile-recompile-cache optimizations. For example, we promisify compilation precisely so that the app can get reasonable behavior even if the code is not cached. If the way this is spec'd necessitates all of us to implement caches or other mitigations then we will have failed our goal of giving people a reasonably portable cost model.

To me the issue is just this: one of the optimizations that we will all effectively have to do for competitive reasons (the 4GB virtual memory bounds checking, which I'll just call the 4GB hack) cannot be done in the current spec without sacrificing one of these things:

You can get away with it if you always allocate 4GB of virtual memory for any wasm memory. This will discourage people from using WebAssembly for small modules, because you will hit virtual memory allocation limits, virtual memory fragmentation, or other problems if you allocate many of these. I also fear that if you allow allocating a lot of them then you will reduce the efficacy of security mitigations like ASLR. Note that existing APIs don't share this danger, since they commit the memory they allocate and they will either OOM or crash before letting you allocate much more than what physical memory allows.
You can get away with it if you allow for a recompile when you find a mismatch during instantiation (compiled code wants 4GB hack but the memory doesn't have that virtual memory allocation). You could also get away with it if instantiation moved the memory into a 4GB region, but see the previous point. So, probably anytime this happens, it'll be a P1 bug for the browser that encountered it.

I think that this means that the spec will encourage vendors to converge to just allowing 4GB reservations anytime wasm memory is allocated, or to have cache/lazy-compile/profile optimizations to detect this.

Finally, I don't understand the point about making any of this non-normative. This can be normative, because we could make the API preclude the possibility of the browser having to compile something without knowing what kind of memory it will have. I imagine that there are many ways to do this. For example, instantiating could return a promise and we could remove the separate compile step. This would make it clear that instantiation is the step that could take a while, which strongly implies to the client that this is the step that does the compilation. In such an API, the compiler always knows if the memory it's compiling for has the 4GB hack or not.

It's sad that we're only noticing this now, but I'm surprised that you guys don't see this is a bigger issue. Is there some mitigation other than caching that I'm overlooking?

@jfbastien

@jfbastien in your motivating scenario, you pointed out that the module was authored to prefer fast memory. I'm assuming you're primarily chasing enabling the fast memory optimization when a particular module wants it, and might be OK with not doing it when the module doesn't want it (nothing bad with opportunistically stumbling upon it in that case, too, just trying to tease apart priorities).

If so, how would these alternatives to caching or to async Instantiate feel like:

Module author must require 4GB as min/max memory
A variant of compile (async at least, maybe also sync) that produces an instance accepting only fast memory.

For the issue of the "4GB hack" and mismatches between memory using it and code expecting it, would it make sense for compilation to internally emit two versions of the code? (Obviously this would use more memory, which is sad, but hopefully compile time wouldn't be much worse, the writer could generate both at once?)

@mtrofin

@mtrofin I don't think it makes sense to ask for 4GiB if you don't intend to use it. The virtual allocation is separate from the use intent, so I think we'd need to separate both.

On 2.: it still isn't super helpful to the developer: if they use that variant and it fails, then what?

@kripken I don't think double compilation is a good idea.

@kripken

@kripken I think that's what we would do without any other resolution to this issue.

I want WebAssembly to be great in the case of casual browsing: you tell me about a cool thingy, send me the URL, I click it, and I amuse myself for a few minutes. That's what makes the web cool. But that means that many compiles will be of code that isn't cached, so compile time will play a big part in a user's battery life. So, double-compile makes me sad.

@mtrofin

@mtrofin

Module author must require 4GB as min/max memory

That's not really practical, since many devices don't have 4GB of physical memory. Also, that's hard to spec.

A variant of compile (async at least, maybe also sync) that produces an instance accepting only fast memory.

I don't think we want double compiles.

@pizlonator

@pizlonator Thus far, we haven't considered designs which required different modes of codegen: we've just always allocated 4gb regions on 64-bit and observed this to succeed for many many thousands of memories on Linux, OSX and Windows. We have a conservative upper bound to prevent trivial total exhaustion of available address space which I expect will be sufficient to support the many-small-libraries use case. So I think the new constraint we're addressing here is that iOS has some virtual address space limitations which could reduce the number of 4gb allocations.

So one observation is that a large portion of the bounds-check-elimination allowed by the 4gb hack can be avoided by just having a small guard region at the end of wasm memory. Our initial experiments show that basic analyses (nothing to do with loops, just eliminating checks on loads/stores w/ the same base pointer) can already eliminate roughly half of bounds checks. And probably this could get better. So the 4gb hack would be a more modest, and less necessary, speedup.

Another idea I had earlier would be to pessimistically compile code with bounds checks (using elimination based on the guard page) and then nop them out when instantiating with a fast-mode-memory. Combined, the overhead could be pretty small compared to idealized fast-mode code.

@lukewagner

@lukewagner

Thus far, we haven't considered designs which required different modes of codegen: we've just always allocated 4gb regions on 64-bit and observed this to succeed for many many thousands of memories on Linux, OSX and Windows. We have a conservative total number to prevent trivial total exhaustion of available address space which I expect will be sufficient to support the many-small-libraries use case. So I think the new constraint we're addressing here is that iOS has some virtual address space limitations which could reduce the number of 4gb allocations.

This isn't an iOS-specific problem. The issue is that if you allow a lot of such allocations then it poses a security risk because each such allocation reduces the efficacy of ASLR. So, I think that the VM should have the option of setting a very low limit for the number of 4GB spaces it allocates, but that implies that the fall-back path should not be too expensive (i.e. it shouldn't require recompile).

What limit do you have on the number of 4GB memories that you would allocate? What do you do when you hit this limit - give up entirely, or recompile on instantiation?

So one observation is that a large portion of the bounds-check-elimination allowed by the 4gb hack can be avoided by just having a small guard region at the end of wasm memory. Our initial experiments show that basic analyses (nothing to do with loops, just eliminating checks on loads/stores w/ the same base pointer) can already eliminate roughly half of bounds checks. And probably this could get better. So the 4gb hack would be a more modest, and less necessary, speedup.

I agree that analysis allows us to eliminate more checks, but the 4GB hack is the way to go if you want peak perf. Everyone wants peak perf, and I think it would be great to make it possible to get peak perf without also causing security problems, resource problems, and unexpected recompiles.

Another idea I had earlier would be to pessimistically compile code with bounds checks (using elimination based on the guard page) and then nop them out when instantiating with a fast-mode-memory. Combined, the overhead could be pretty small compared to idealized fast-mode code.

Code that has bounds checks is best off pinning a register for the memory size and pinning a register for memory base.

Code that uses the 4GB hack only needs to pin a register for memory base.

So, this isn't a great solution.

Besides the annoyance of having to wrangle the spec and implementations, what are the downsides of combining compilation and instantiation into one promisified action?

FWIW, Unity 5.6 will use WebAssembly.instantiate. jonas

…

On Dec 28, 2016, at 11:42 PM, Seth Thompson ***@***.***> wrote: I've been meaning to provide a better example for why we need to explicitly discourage (or, as I've argued in the past, even disallow) synchronous compilation/instantiation. Hopefully this provides additional fodder for discussion. Let's zoom out and talk about user experience for a second. Here's a comparison between loading the AngryBots demo with asynchronous calls (via asm.js), left, and synchronous calls (current WebAssembly implementation), right. Synchronous compilation provides a terrible user experience and breaks progress bars or any other visual indication of application loading. Whatever the effort, I think we need to work together to make sure that Emscripten, Unity, and other tools export asynchronously loading WebAssembly by default. cc @jechter @juj Not only that, I think we need to make it prohibitively difficult for developers to fall into the trap of writing synchronous loading code. We need to explicitly encourage WebAssembly.compile & WebAssembly.instantiate and discourage new WebAssembly.Module & new WebAssembly.Instance. Let's dive a little deeper and see how bad that progress bar on the right is, for WebAssembly. Here's a trace captured with the DevTools performance panel: It takes 30 seconds on my MacBook Air to show any progress bar at all. What takes that time? Let's zoom into the ~20s after the wasm module has downloaded when the main thread is totally locked up: Compilation is taking ~20s and instantiation is taking ~2s. Note that in this case, the Unity loader is already calling the asynchronous WebAssembly.compile if it's supported, so the 20s is because V8 still does a synchronous compile under the hood. cc @titzer @flagxor this is something V8 really needs to fix. The 2s instantiation jank is due to the Unity loader code calling the synchronous new WebAssembly.Instance. This is what we really need to fix in both Unity code and Emscripten. I think it's also worth mentioning that this is not merely the risk of a developer shooting themselves in the foot or not. The average web page includes dozens of third party scripts: those scripts all have the potential to include top-document-janking WebAssembly. (If you'd like to explore the trace in more detail you can view the full timeline here: https://chromedevtools.github.io/timeline-viewer/?loadTimelineFromURL=https://www.dropbox.com/s/noqjwql0pq6c1wy/wasm?dl=0) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

@jfbastien

@jfbastien Maybe I don't understand what you mean by "conclusions". I am mostly just presenting options here. I don't have a conclusion myself.

If we want the Memory-at-Compile-time opt right now, I proposed a few options that can let us have it ASAP, trivially modifying our current sync compilation.

Or if we want that opt now in an async way, I proposed an option ("1 + 3", i.e., not receiving all the imports at compile time, just the Memory) that can let us have it ASAP as well.

Or if we want to focus now on an async version of that opt with the current API (not "1 + 3") then we can get that too, it will just take some careful planning because it would be a breaking change. I don't think it's an option for us here to decide to break existing users, so we would need to consult the community. Possibly there will be few concerns and we can just do it, in which case, how much effort depends on how much overhead we are willing to tolerate - if we cannot accept any overhead at all, then it might be a lot. Or possibly there will be concerns, which is my personal intuition - any breaking change must be done gradually - in that case it will take longer, we'd probably start with a new option and eventually plan to make it the default.

Again, no conclusions from me. All the above are options. It really depends what you care about more: Memory-at-Compile-time, or async, or both; and the issue of tolerating new overhead vs not; and whether you want this urgently or can wait, etc. etc. I'm happy to help on the emscripten side with any of the above, however people decide.

Or if we want to focus now on an async version of that opt with the current API (not "1 + 3") then we can get that too, it will just take some careful planning because it would be a breaking change. I don't think it's an option for us here to decide to break existing users, so we would need to consult the community. Possibly there will be few concerns and we can just do it, in which case, how much effort depends on how much overhead we are willing to tolerate - if we cannot accept any overhead at all, then it might be a lot. Or possibly there will be concerns, which is my personal intuition - any breaking change must be done gradually - in that case it will take longer, we'd probably start with a new option and eventually plan to make it the default.

To be sure I understand:

What are the overheads?

Are these overheads inherent to Async + Memory-at-compile-time, or are they implementation constraints which can be fixed later? IMO if they're inherent to A+M then we should fix the API ASAP.
IIUC the overheads are a temporary thing on either imports or exports, and aren't inherent to A+M, is that correct? Could you detail this a bit more?

Which users are we talking about breaking? asm.js users who want the same code to also target wasm?

Put another way: what's the ideal end-state for WebAssembly if we had infinite time and no "legacy"? I think we've designed for that ideal end-state, and you're pointing out hurdles on the way, but I'm still not sure that's the case. If so, then I'm not sure I have enough information to figure out if WebAssembly needs a stopgap to ease transition, or if temporary burden is acceptable for a subset of Emscripten users (and which users).

About the overheads in some of the options: As described above the tricky thing is we must send the imports and receive the exports synchronously currently, and that we don't have the JS imports until the JS is ready (but we do have the Memory!). One way to get around that is to add a layer of indirection on either the imports or exports. For example, we could provide thunks instead of the true imports very early (in the HTML, before the JS), so we appear to provide them synchronously (but it's just the thunks, which are trivial to create even before we have the JS). Then later when we have the JS, we can update the thunks to point to the right place. This would add overhead of another JS call and a JS object lookup for each import call. Also some overhead in code size and startup.

About breaking users: We want existing emscripten users to be able to flip a flag and get working wasm. Many users and partners are happy about that. It lets them compare asm.js and wasm and it lets projects they spent effort on porting benefit from wasm with no new porting effort. And it avoids the risk that if they also need to asyncify their startup code while porting to wasm, and something breaks, they might blame wasm unnecessarily.

@s3ththompson

that we don't have the JS imports until the JS is ready (but we do have the Memory!)

That seems incidental and not something we should bake into time forever with an API. Furthermore, for the reasons @s3ththompson explained, even ignoring the memory-at-compile-time issue, we need instantiation to be async anyway. So I agree with @jfbastien that we have our ideal end-state API here and that should not be what we compromise on here.

About breaking users: if we offer a simple migration path (like an "onload" event/promise), it should be simple for users to migrate, with the carrot being all the major perf wins. I don't think we have any evidence that exact asm.js source backcompat is a hard requirement for the default configuration; we have many examples of quite the opposite: users willing to do whatever to achieve the optimal perf since that is, of course, the whole point.

@lukewagner

Talking to @lukewagner offline, we think the best thing is to start with enabling instantiate for wasm in the default recommended setting in emscripten, by adding an indirection layer (as described above, but on the exports). It will probably have negligible overhead for most use cases. So this gets us the benefits we all want here.

Eventually, we can get rid of that small overhead, but it will be a breaking change that will require a bunch more planning and mailing list discussion etc., as there isn't an obviously good/right way to do it, we realized. But since we think the overhead of the indirection layer is very low, this part isn't urgent.

@kripken

@kripken great! It would be helpful to have a diagram showing different instances / JS / imports / exports / memory / tables. Do you want to move this discussion elsewhere (Emscripten / binaryen repo)? I have a mental picture of what I think C++ code should be organized at, but it's pretty obvious by now that you don't have the same picture! You've got more experience there, so I'd love to learn from it and help however I can.

@jfbastien

@jfbastien: sure. I'm not clear yet on what you're looking for in a diagram, but yeah, maybe another place is better. For the implementation of this in emscripten there is the issue mentioned before, emscripten-core/emscripten#4711, also feel free to open another if that doesn't cover what you want.

IIUC Emscripten uses this by default now. Closing.

@s3ththompson

To follow-up on @s3ththompson comment, note that synchronous compilation and instantiation is useful. Notably, I recently ran into a situation where I wanted to synchronously load and compile a WebAssembly module in Node.js (v7.7.2). If only APIs which return promises are available, this means that I would be unable to provide a synchronous export.

When deciding whether to provide either async, sync, or both APIs, remember that a browser context is not the only environment in which people want to use WebAssembly. A number of people, myself included, are interested in WebAssembly's potential as an efficient compile target combined with the Node.js runtime, akin to the JVM. While async import/exports may land in Node.js, synchronous import and export will remain the dominant pattern for the foreseeable future. In which case, the ability to load a WebAssembly module and synchronously compile and instantiate that module is highly useful.

@kgryte

@kgryte I should have clarified that my comment pertained primarily to the browser as an execution context. We've landed on an API surface that still exposes the synchronous APIs. Browsers may impose a size limit on modules passed to the synchronous APIs (Chrome already does, for example), but that limit is configurable by the embedder and should not need to apply to Node.

@s3ththompson

@s3ththompson Thanks for the clarification.

Navigation Menu

Repository navigation

Can WebAssembly.Instance recompile an already-compiled Module? #838

67 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Can WebAssembly.Instance recompile an already-compiled Module? #838

Description

Activity

mbebenita commented on Oct 27, 2016

rossberg commented on Oct 27, 2016

lukewagner commented on Oct 27, 2016

jfbastien commented on Oct 27, 2016

lukewagner commented on Oct 27, 2016

jfbastien commented on Oct 27, 2016

lukewagner commented on Oct 27, 2016

pizlonator commented on Oct 27, 2016

mtrofin commented on Oct 27, 2016

kripken commented on Oct 27, 2016

jfbastien commented on Oct 27, 2016

pizlonator commented on Oct 27, 2016

pizlonator commented on Oct 27, 2016

lukewagner commented on Oct 27, 2016

pizlonator commented on Oct 27, 2016

67 remaining items

jechter commented on Dec 29, 2016

kripken commented on Dec 30, 2016

jfbastien commented on Dec 30, 2016

kripken commented on Dec 30, 2016

lukewagner commented on Dec 30, 2016

kripken commented on Dec 30, 2016

jfbastien commented on Dec 30, 2016

kripken commented on Dec 31, 2016

jfbastien commented on Feb 6, 2017

kgryte commented on Mar 11, 2017

s3ththompson commented on Mar 13, 2017

kgryte commented on Mar 13, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions