Repository navigation
Checking in .bin.wast #119
Description
In maintaining https://github.com/titzer/wizard-engine, we made the decision to only parse .wast tests where the modules themselves are in binary, i.e. .bin.wast. This has been very helpful in avoiding both the work and potential incompatibilities with implementing our own parser for the entire text format. To date, Wizard's strategy has been to check out the spec repo and proposal repos, building their respective interpreters, and then generate the .bin.wast.
As it turns out, as proposals either stagnate or get their tests merged into the main spec, their respective repos get stuck with their respective toolchains (to wit, old versions of Ocaml, ocamlbuild, or dune).
I was wondering if there is any appetite for building the spec interpreter for each of the tracked proposals and then also building the .bin.wast outputs of them, and checking them in here automatically as part of the CI.
Activity
alexcrichton commentedon Feb 6, 2025
Personally I like the idea of lowering the barrier to entry to run the tests here, although if CI goes so far as to convert to
*.bin.wastI'd say that it might be better to go as far as something like the output ofwast2jsonwhere it's binaries + json which is, in principle, even easier to parser. That assumes that using your local language's json parser is likely easier than writing an s-expression parser, even if you're just parsing binary modules and not each individual instruction for example.To me though there's a lot of downsides in these ideas though. Building the spec interpreter in CI here is probably going to be very nontrivial (especially if each repo needs its own version of ocaml/dune/etc). Running something like
wast2jsonis also nontrivial because it means that some tool (e.g.wast2jsonitself or some other tool likewasm-tools json-from-wast) has to support proposals before being integrated with this repository, which also isn't great. And there's also the issue of the exact json schema etc etc.One strong upside of this repo as-is is that the maintenance burden is quite low. Postprocessing
*.wastis going to increase that no matter what and to me it'll be a game of minimizing that cost as much as possible.dschuff commentedon Feb 6, 2025
Agreed that this would be nontrivial. I'll also add thought that something like this would be useful for importing wasm tests into WPT (Web Platform Tests). We currently have automation that uses the interpreter to convert the wast tests into binary and generate HTML/JS tests in the WPT style and import them. This currently works for things that get merged into the main branch but it would be nice to get them from proposals too, and there we'd have the same problem of needing either the interpreter from that branch, or another tool like Binaryen or wasm-tools.
titzer commentedon Mar 3, 2026
@alexcrichton I think running the reference interpreter to generate
.bin.wastin CI isn't too big of a burden, these are the lines in Wizard's CI:tlively commentedon Mar 3, 2026
If we can generate .bin.wast without installing multiple versions of OCaml and its packages, I would be in favor of doing so.
rossberg commentedon Mar 3, 2026
FWIW, in the spec repo itself it's now just these actions to build the reference interpreter:
Edit: In your case,
depsinstead ofjsdepsis probably enough.alexcrichton commentedon Mar 3, 2026
My main worry w.r.t. ocaml is that each proposal might need a different version of ocaml if the spec interpreter ever updates. This is trying to pull together a whole bunch of proposal repos into one repo and they've all got their own version of the spec interpreter and different merge-base's with the main spec. I don't doubt that building one interpreter and getting some deps out isn't too bad, it's more the constellation of spec repos and getting them all to line up. That being said, though, I don't think it's problematic to at least try to generate
*.bin.wastfor the files we can, even if it can't cover everything. Getting the main body of tests I think would still be a good win.Orthogonal to that though, do y'all have thoughs a JSON-style approach? If the goal is to ingest an easier-to-parse form than the raw text format itself then I would suspect that parsing JSON is far easier than parsing
*.bin.wast. I don't doubt that parsing*.bin.wastis easier than parsing the tests as-is, but if we're going to far as to make it easier for tools to ingest then this seems worthwhile to consider. I understand the spec interpreter does not have support for this right now, but that's something that seems tractable as I assume ocaml has some form of JSON serializer library to use.rossberg commentedon Mar 3, 2026
I don't think you should ever need multiple versions of ocaml. The max required one should do, it's very backwards compatible.
tlively commentedon Mar 4, 2026
Assuming that the JSON output would be a new feature built into the reference interpreter, it sounds like we would need to get the reference interpreter built on CI here either way. How about we start with the .bin.wast since the interpreter already supports that, then we can additionally add some JSON dump format once the interpreter supports that as well? I guess the only downside would be having more formats than we need at that point, but if all downstreams prefer the JSON, then we could just remove the .bin.wast.
rossberg commentedon Mar 4, 2026
https://seriot.ch/software/parsing_json.html — just sayin', since this thread is replicating the myth of JSON's alleged simplicity and portability.
titzer commentedon Mar 22, 2026
I generally agree that adding JSON to the mix, even though it is already necessarily supported by web engines, doesn't make things easier for other engines. As it is,
.wastis already complete, and (in my view).bin.wastis a simplification of that format to just assertions and binaries. Even in the Claude era, fielding a (new) JSON parser for this purpose seems like overkill.