Test citgm execution on RISC-V

[sxa]

We've got unofficial builds working well on RISC-V. It would be good to check the state of the module ecosystem on Linux/riscv64 with citgm.

I've got a couple of runs taking place on my boards at the moment and will add any findings into here.

Related to nodejs/build#4099

(Creating this today after seeing a post elsewhere about the module ecosystem on RISC-V from @gounthar - ref: https://fosstodon.org/@sxa/116979881114021674)

[sxa]

Full output is attached below but a quick summary from egrep '<testcase|<failure' report.xml | grep -B1 failure is here:

sxa@bananapif3:~/citgm$ egrep '<testcase|<failure' report.xml  | grep -B1 failure
  <testcase name="@nearform/bubbleprof-v7.0.1" time="469.55">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="@nearform/doctor-v8.1.0" time="402.061">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="acorn-v8.17.0" time="648.009">
    <failure message="module test suite failed">Error: Install Timed Out </failure>
  <testcase name="async-v3.2.6" time="198.488">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="ava-v8.0.1" time="743.554">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="bcrypt-v6.0.0" time="76.464">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="bluebird-v3.7.2" time="133.99">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="bufferutil-v4.1.0" time="42.832">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="cheerio-v1.2.0" time="118.806">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="clinic-v13.0.0" time="841.885">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
  <testcase name="coffeescript-v2.7.0" time="206.433">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="crc32-stream-v7.0.1" time="27.484">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="ember-cli-v7.1.0" time="17.29">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="fastify-v5.10.0" time="609.652">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="gulp-v5.0.1" time="148.596">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="https-proxy-agent-v9.1.0" time="6.951">
    <failure message="module test suite failed">Error: incorrect header check </failure>
  <testcase name="iconv-v3.0.1" time="17.515">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="inherits-v2.0.4" time="154.239">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="jest-v30.4.2" time="198.903">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="jquery-v4.0.0" time="157.539">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="level-v10.0.0" time="214.836">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="leveldown-v6.1.1" time="242.553">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="microtime-v3.1.1" time="168.072">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="nan-v2.28.0" time="68.836">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="node-gyp-v13.0.1" time="756.558">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
--
  <testcase name="npm-v12.0.1" time="935.28">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
--
  <testcase name="pino-v10.3.1" time="925.203">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
--
  <testcase name="q-v1.5.1" time="55.466">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="rewire-v9.0.1" time="85.835">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="rimraf-v6.1.3" time="302.082">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="router-v2.2.0" time="111.741">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="semver-v7.8.5" time="321.118">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="serialport-v13.0.0" time="197.642">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="sqlite3-v6.0.1" time="146.222">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="tape-v5.10.2" time="234.425">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="tough-cookie-v6.0.2" time="80.153">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="undici-v8.9.0" time="730.575">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
  <testcase name="uuid-v14.0.1" time="140.021">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="winston-v3.19.0" time="180.831">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="ws-v8.21.1" time="105.564">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="yargs-v18.0.0" time="276.612">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="yeoman-generator-v8.2.2" time="513.365">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
sxa@bananapif3:~/citgm$ 

report.xml.gz

nohup.out.gz

[sxa]

And from a run on a K3:

  <testcase name="@hapi/shot-v6.0.3" time="9.629">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="@nearform/bubbleprof-v7.0.1" time="13.794">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="@nearform/doctor-v8.1.0" time="14.69">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="@nearform/flame-v9.1.0" time="8.515">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="acorn-v8.17.0" time="293.255">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="async-v3.2.6" time="64.451">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="ava-v8.0.1" time="422.14">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="bcrypt-v6.0.0" time="12.576">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="bluebird-v3.7.2" time="38.148">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="bufferutil-v4.1.0" time="13.053">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="cheerio-v1.2.0" time="19.798">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="clinic-v13.0.0" time="11.57">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="coffeescript-v2.7.0" time="62.757">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="crc32-stream-v7.0.1" time="6.909">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="ember-cli-v7.1.0" time="4.581">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="fastify-v5.10.0" time="14.583">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="https-proxy-agent-v9.1.0" time="1.844">
    <failure message="module test suite failed">Error: incorrect header check </failure>
  <testcase name="iconv-v3.0.1" time="4.377">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="inherits-v2.0.4" time="57.783">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="jest-v30.4.2" time="53.754">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="jquery-v4.0.0" time="38.314">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="level-v10.0.0" time="66.67">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="leveldown-v6.1.1" time="71.575">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="microtime-v3.1.1" time="44.106">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="nan-v2.28.0" time="20.137">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="node-gyp-v13.0.1" time="644.795">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
--
  <testcase name="npm-v12.0.1" time="317.61">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="pino-v10.3.1" time="420.275">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="prom-client-v15.1.3" time="13.807">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="q-v1.5.1" time="17.907">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="rewire-v9.0.1" time="20.876">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="rimraf-v6.1.3" time="142.527">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
  <testcase name="router-v2.2.0" time="35.687">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="semver-v7.8.5" time="8.405">
    <failure message="module test suite failed">Error: Install Failed </failure>
  <testcase name="serialport-v13.0.0" time="40.747">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="sqlite3-v6.0.1" time="43.324">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="tape-v5.10.2" time="18.082">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="through2-v5.0.7" time="8.122">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="tough-cookie-v6.0.2" time="19.006">
    <failure message="module test suite failed">Error: The canary is dead: </failure>
--
  <testcase name="undici-v8.9.0" time="629.679">
    <failure message="module test suite failed">Error: Test Timed Out </failure>
  <testcase name="uuid-v14.0.1" time="30.483">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="ws-v8.21.1" time="22.74">
    <failure message="module test suite failed">Error: Install Failed </failure>
--
  <testcase name="yeoman-generator-v8.2.2" time="135.028">
    <failure message="module test suite failed">Error: The canary is dead: </failure>

report.xml.gz

nohup.out.gz

[gounthar]

I went through the two report.xml and nohup pairs you attached. One thing stands out that I think makes a chunk of the table unreliable, though I might be misreading how you invoked it.

Every native addon fails at node-gyp configure, before compiling anything:

gyp WARN read config.gypi ENOENT: ... open '/home/iojs/citgm/include/node/config.gypi'
gyp: /home/iojs/citgm/common.gypi not found (cwd: .../bcrypt) while reading includes of binding.gyp

npm_config_nodedir is pointing at the citgm checkout itself, /home/sxa/citgm on the F3 and /home/iojs/citgm on the K3, and neither contains common.gypi or include/node/config.gypi. citgm only sets that variable when you pass -d/--nodedir, and it resolves relative to cwd (lib/create-options.js), so citgm-all -d . from the citgm directory would produce exactly this. 17 occurrences in the F3 run, 21 in the K3 one, covering bcrypt, bufferutil, iconv, level/classic-level, leveldown, microtime, serialport (@parcel/watcher), sqlite3 and ws. Dropping -d, or pointing it at real headers, should clear that whole class.

Two smaller things from the same logs. uuid looks like a genuine riscv64 gap: it dies in lefthook/get-exe.js, which has no riscv64 binary to find. ember-cli and q are architecture-independent, EUNSUPPORTEDPROTOCOL on a workspace: dependency and a grunt postinstall exiting 3. The F3 run also hit ENOSPC: no space left on device at least once, so one result there is void regardless.

Probably worth saying for anyone reading those tables that "The canary is dead:" is just citgm's message for npm test exiting non-zero (lib/package-manager/test.js), not a Node crash. It reads far more alarming than it is.

One last thing I wasn't sure about: both runs use the pointer-compression build. Is that deliberate? If the aim is to characterise the module ecosystem rather than that specific configuration, a stock riscv64 build might be the fairer baseline, but you may well have a reason for it that I'm missing.

[sxa]

I'll be honest I just uploaded the files and hadn't made any real effort to diagnose them yet due to other higher priority stuff so thanks for the initial look! Yeah sounds like I didn't have the headers available in the right place as you say.
An absence of RISC-V binaries isn't a surprose for some modules but it'll be useful to see which modules are in that category so that's good to see.
The unofficial builds are currently all with pointer compression enabled, but I can easily test with a build with it enabled if we think it'll make a difference to any of the tests. There is an issue around from someone who wanted builds with it enabled and RISC-V seemed like a reasonable choice of build to have it switched on. But honestly, the main reason I 'm building with it enabled by default was mainly because I do run a personal Node.js+MySQL server on a D1 board with very limited RAM so having pointer compression will help with the memory usage, so it's mostly for selfish reasons ;-)

if we ever got RISC-V promoted to be a tier 2 platform we can make a decision on whether to have it on or off. I did initially have the unofficial builds system configured to have a GCC build without PC and a clang build with PC (e.g. 25.9.0) but no-one has objected to it being on from when I enabled it. That might change once the server-class boards start to become readily available.

[sxa]

Updated version from the K3:

• report.xml.gz
• nohup.out.gz

Updated results from BPi-F3:

• report.xml.gz
• nohup.out.gz