Skip to content

EMSDK support for wasm backend #6350

Closed
Closed

Description

@jgravelle-google

As part of using the wasm backend by default, we should make it easy for users to download and install it. This means having emsdk install upstream LLVM instead of fastcomp.

Steps:

  1. Add wasm backend to emsdk behind a flag
  2. Switch to wasm backend by default (fastcomp behind a flag)
  3. Deprecate fastcomp backend

There's the question of how to organize installing the two backends. The options I know of are

  1. Download only one backend
    • need to rerun emsdk install to download the other backend later (which is bound to be a frustrating UX)
    • need to emsdk activate to change backends
  2. Download both backends
    • no need to download/compile the other backend later, but instead frontloads that cost
    • may need to edit ~/.emscripten to change the LLVM_ROOT variable
  3. Download one LLVM that has both backends enabled
    • integrate upstream changes into fastcomp branch to get latest wasm backend, build fastcomp with js and wasm32 targets enabled
    • need to merge regularly going forward
    • changing backends is a matter of changing ~/.emscripten to set... something. Currently EMCC_WASM_BACKEND is an environment flag, can .emscripten set that?

Activity

  1. kripken commented on Jan 25, 2019

    @kripken
    Member

    Current status is that the emsdk can install latest-upstream which gets the latest lkgr. Users can install both that and fastcomp, and switch between them.

    Not yet done:

    • non-linux
    • emcc version integration, i.e., something other than "last lkgr" (still thinking about what to do with this in our testbots and builders)
  2. DragonOsman commented on May 9, 2019

    @DragonOsman

    (I'm posting this question here, but also wondering if it's okay for me to do so)

    So this doesn't support Windows yet (at least at the time of writing)? And am I correct in assuming that when it's complete, developers would be able to use the WASM Backend through EMSDK? What about the global Node installation? I have one for the latest version of Node, and I want to use that with EMSDK if possible, preferably without having to keep changing the .emscripten file every time I update the EMSDK (both for the path to the Node installation and for the LLVM root setting).

    And if it still doesn't support Windows yet, does anyone have any idea when it might be able to? Just wondering. Thanks.

  3. kripken commented on May 10, 2019

    @kripken
    Member

    Yes, eventually the wasm backend will replace the older backend, and builds will be available on all platforms. Meanwhile, you can build it from source, or if some other place has recent LLVM builds you can use those (since it's just a normal LLVM build - for example, maybe wasi has builds for windows).

  4. DragonOsman commented on May 13, 2019

    @DragonOsman

    Right now the wasm backend seems to only work on Linux distributions. Lately, when I try to do it on Windows, I get an error from gen_struct_info.py. Invoking Node gives a non-zero value, with Node itself just giving an empty output instead of some sort of message. It used to work before, so I don't know what happened. So right now it seems like I'll have to use the WSL if I want to do it on Windows.

  5. DragonOsman commented on May 13, 2019

    @DragonOsman

    This is the error message I get when trying to use the wasm backend with Emscripten to compile C or C++ code to WebAssembly:

    cache:INFO: generating system asset: generated_struct_info.json... (this will be cached in "C:\Users\Osman\.emscripten_cache\wasm-obj-pic\generated_struct_info.json" for subsequent builds)
    Traceback (most recent call last):
      File "C:\emsdk\emscripten\incoming\emcc.py", line 3334, in <module>
        sys.exit(run(sys.argv))
      File "C:\emsdk\emscripten\incoming\emcc.py", line 1947, in run
        final = shared.Building.emscripten(final, target + '.mem', js_libraries)
      File "C:\emsdk\emscripten\incoming\tools\shared.py", line 2204, in emscripten
        emscripten.run(infile, outfile, memfile, js_libraries)
      File "C:\emsdk\emscripten\incoming\emscripten.py", line 2560, in run
        shared.Settings.STRUCT_INFO = shared.Cache.get(generated_struct_info_name, ensure_struct_info)
      File "C:\emsdk\emscripten\incoming\tools\cache.py", line 133, in get
        temp = creator()
      File "C:\emsdk\emscripten\incoming\emscripten.py", line 2557, in ensure_struct_info
        gen_struct_info.main(['-qo', out, path_from_root('src', 'struct_info.json')])
      File "C:\emsdk\emscripten\incoming\tools\gen_struct_info.py", line 529, in main
        struct_info = inspect_code(header_files, cpp_opts, structs, defines)
      File "C:\emsdk\emscripten\incoming\tools\gen_struct_info.py", line 407, in inspect_code
        info = shared.run_js(js_file[1]).splitlines()
      File "C:\emsdk\emscripten\incoming\tools\shared.py", line 1090, in run_js
        return jsrun.run_js(filename, engine, *args, **kw)
      File "C:\emsdk\emscripten\incoming\tools\jsrun.py", line 155, in run_js
        raise Exception('Expected the command ' + str(command) + ' to finish with return code ' + str(assert_returncode) + ', but it returned with code ' + str(proc.returncode) + ' instead! Output: ' + str(ret)[:error_limit])
    Exception: Expected the command ['C:/Program Files/nodejs/node.exe', 'c:\\users\\osman\\appdata\\local\\temp\\tmpzu370u.js'] to finish with return code 0, but it returned with code -1073741819 instead! Output:
    
    

    Note the empty output from Node, as well as the other stuff mentioned in the exception.

    Do I need to debug the Python script?

  6. kripken commented on Jun 1, 2019

    @kripken
    Member

    I think we can close this issue. At this point we have wasm backend support in emsdk for windows, mac and linux, and tested on our new CI, all installable with ./emsdk install latest-upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions