[Feature Request] Support building for both UMD/ESM

When building wasm, I have to choose one between UMD and ESM for the generated JavaScript file, by either specifying the extension (.js vs .mjs) or using -sEXPORT_ES6=1. I want to build both of them but seems no way to do that.

Building it twice is not a working workaround because some number in the generated ASM_CONSTS table is different in 2 builds and the generated 2 JS files cannot use with one .wasm file.

How hard do you think it would be to make a module that works in both of these environments? Is it worth adding yet anther setting or should we just try to make the ES6 output UMD compatible?

Out of interest what is the platform you are targeting that requires UMD compat?

How hard do you think it would be to make a module that works in both of these environments? Is it worth adding yet anther setting or should we just try to make the ES6 output UMD compatible?

Out of interest what is the platform you are targeting that requires UMD compat?

I don't think the idea that one module supporting both UMD and ESM can work, because reference to import (eg. import.meta.url) will cause an syntax error if not in ESM.

The reason why I want to distribute a UMD bundle is because of bundler compatibility. There are 2 reasons:

A ESM module can import a CJS module and bundler can do this correctly; but it is very hard to import ESM in a CJS module.
Currently some bundlers (Webpack, parcel, ..) will rewrite import.meta.url into static strings like "file:///path/to/build/time/file.js". This will make the Emscripten generated code not working.

As a library author, the best option for me is to offer 2 exports - one for UMD and one for ESM. This is why I created this issue.

BTW it would be great if Emscripten can support generate multiple targets at one build (not only UMD/ESM). In my use case, I acutally need 4 targets:

output.js (umd, ENVIRONMENT=web,webview,worker)
output.mjs (esm, ENVIRONMENT=web,webview,worker)
output.node.js (umd, ENVIRONMENT=node)
output.node.mjs (esm, ENVIRONMENT=node)

I am currently using Regex to replace the code snippet in generated files in order to exclude Node.js outputs for web. This is very hacky and unstable because with the upgrade of emscripten the old regex may no longer work.

You may be interested in why I don't use default. Because default supports both web and node, right? The reason is the bundler again:

Bundler is not happy with await import("module") even it is guarded by condition isNode.
Tree shaking is not working with the variable of IS_ENVIRONMENT_NODE

I would share a viewpoint from a library developer: some requirement above is not a problem of Emscripten, they are more like the problem of bundler. However, we cannot control or predict how users use bundler. If bundler has a bug or an unexpected behavior, we can track the bugfix but at the same time we need to workaround to offer user the out-of-box experience.

@sbc100

@sbc100 could you help to take a look for this feature request? I tried a few ways in my library but it seems that this cannot be workaround.

@fs-eire

@fs-eire as a workaround are you able to link your project N times in order to get N different wasm/js files? i.e. are you just looking to speed up the build?

I guess my question is really, can you ship 4 different wasm files to go along with your 4 different js files?

It doesn't work because the ASM_CONSTS table (and other build time generated things) may be different for each build. There is a step that using wasm-opt to strip the unused exports and that step applied to both .wasm and .js, which almost generate different contents every time.

This causes mismatch for the .wasm and .js if build multiple times.

EDIT: this step: https://github.com/emscripten-core/emscripten/blob/main/tools/link.py#L2218-L2221

It doesn't work because the ASM_CONSTS table (and other build time generated things) may be different for each build. There is a step that using wasm-opt to strip the unused exports and that step applied to both .wasm and .js, which almost generate different contents every time.

This causes mismatch for the .wasm and .js if build multiple times.

EDIT: this step: https://github.com/emscripten-core/emscripten/blob/main/tools/link.py#L2218-L2221

Right, but if you build and ship separate wasm and js files for each configuration then it should work find right? (i.e. if you never share wasm files between different JS files).

It doesn't work because the ASM_CONSTS table (and other build time generated things) may be different for each build. There is a step that using wasm-opt to strip the unused exports and that step applied to both .wasm and .js, which almost generate different contents every time.
This causes mismatch for the .wasm and .js if build multiple times.
EDIT: this step: https://github.com/emscripten-core/emscripten/blob/main/tools/link.py#L2218-L2221

Right, but if you build and ship separate wasm and js files for each configuration then it should work find right? (i.e. if you never share wasm files between different JS files).

Do you mean that I have to deploy:

file_0.wasm
file_0.mjs <--- ESM
file_1.wasm
file_1.js <--- UMD

This is not what I want. Not only it doubles the size of the packages, but also makes it hard for users to deploy and very difficult to troubleshooting (if there is a mismatch).

Yes that is what I mean. Sharing a single wasm file between different JS files built with different settings is not supported, so as a fallback I think can ship N * wasm files and N * JS files such that each JS matches a specific wasm. In fact perhaps you could ship N different NPM packages e.g. mypackage-esm + mypackage-umd ?

Could you use conditional package exports?

Could you use conditional package exports?

https://nodejs.org/api/packages.html#conditional-exports

https://webpack.js.org/guides/package-exports/

I already uses this. The problem is I cannot have 2 different JS files (ESM/UMD) for the same wasm file.

Yes that is what I mean. Sharing a single wasm file between different JS files built with different settings is not supported, so as a fallback I think can ship N * wasm files and N * JS files such that each JS matches a specific wasm. In fact perhaps you could ship N different NPM packages e.g. mypackage-esm + mypackage-umd ?

I really don't want to do this. So currently I build esm only and uses dynamic import import('./a.mjs') in both my UMD/ESM bundle. However this creates some new problems - some bundlers are not happy with the dynamic import.

The problem is that, as you have found, any emscripten command line flag changing can result in the different wasm binary. Building a single wasm along with N different JS files seems like something we are unlikely to do, given how much complexity it would add. It would be pretty big change I believe.

Could you store each build in its own subfolder?

Something like this?

{
  "type": "module",
  "exports": {
    "node": {
      "module": "./node-esm/index.js",
      "require": "./node-umd/index.cjs"
    },
    "default": "./web-esm/index.js"
  }
}

The problem is that, as you have found, any emscripten command line flag changing can result in the different wasm binary. Building a single wasm along with N different JS files seems like something we are unlikely to do, given how much complexity it would add. It would be pretty big change I believe.

Yes, you are right. I tried to make a local change to add -sES6=2 for building both UMD (target.js) and ESM (target.mjs), and it turns out to be quite a big change... But I think this is still the easiest way to support this feature request. Seems no easy way to do this.

And just do confirm, as far as you are aware there is no way to satisfy both ESM and UDM requirements in the single JS file?

I think building just the ESM version and then writing some sed scripts the update it might be the simplest solution.. is that what you are doing today?

And just do confirm, as far as you are aware there is no way to satisfy both ESM and UDM requirements in the single JS file?

I think building just the ESM version and then writing some sed scripts the update it might be the simplest solution.. is that what you are doing today?

I thought about this too. But closure compiler makes it almost impossible to do that.

If I disable the closuer compiler, I can use some string replace to make it work. However, I don't do in this way outside of Emscripten because every Emscripten version upgrade may possibly break it.

I am not using this solution today. I uses dynamic import (the import() function) to import ESM in UMD. The problem is I cannot make a single bundle file for UMD and I already get complaint about it today.

there is no way to satisfy both ESM and UDM requirements in the single JS file

There is no way. import/export statement and import.meta are syntax error for UMD.

The major differences between the generate JS are:

the way how it determine the script URL. UMD uses relatively complicated way (document.currentScript.src or self.location.href in web, and __filename in node), and ESM uses import.meta.url.
import/export. UMD uses require() and ESM uses import statement; UMD uses module.exports and ESM uses export statement.
- In Node.js+ESM, require is not available but code need it. so it is generated from import {createRequire} from 'module'; var require = createRequire(); for Node.js only build. However, for Node.js + web build (this is default), top-level import for 'module' cannot work in web, so uses dynamic import instead: if (IS_NODE) { const {createRequire} = await import('module'); var require = createRequire(); }. This also added top-level await, which makes closure compiler not working. The current Emscripten uses a workaround: run closure compiler first and then added the top-level await into the JS code. This is the major reason why bundler not happy with it.
worker related codes. Including some PThread related code and the worker creation.

And just do confirm, as far as you are aware there is no way to satisfy both ESM and UDM requirements in the single JS file?

I think the new Node API process.getBuiltinModule(id) can help with this by removing the need to use require or import('module') to load the "fs", "path", and "url" modules.

Doc: https://nodejs.org/docs/latest-v20.x/api/process.html#processgetbuiltinmoduleid

Maybe the code could use this when the min Node version is 20.16 or greater.

nvtt: posh.h — switch EMSCRIPTEN macro to __EMSCRIPTEN__

nvtt: posh.h — switch EMSCRIPTEN macro to __EMSCRIPTEN__ (#126)

Navigation Menu

Repository navigation

[Feature Request] Support building for both UMD/ESM #21899

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Feature Request] Support building for both UMD/ESM #21899

Description

Activity

sbc100 commented on May 7, 2024

fs-eire commented on May 7, 2024

fs-eire commented on May 7, 2024

fs-eire commented on May 7, 2024

fs-eire commented on Jun 10, 2024

sbc100 commented on Jun 10, 2024

sbc100 commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

sbc100 commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

sbc100 commented on Jun 10, 2024

dasa commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

sbc100 commented on Jun 10, 2024

dasa commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

sbc100 commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

fs-eire commented on Jun 10, 2024

dasa commented on Jul 24, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions