vitest fails to load typescript worker code in node (but does so correctly in browser mode) #5757

Activity

1. andreashuber69

   added

   pending triage

   on May 21, 2024
2. hi-ogawa commented on May 22, 2024

   

   hi-ogawa

   on May 22, 2024

   Last edited by hi-ogawa

   Collaborator

   More actions

   Browser mode is quite different and it just happens that Vite's web worker support makes it work, but I don't think the same technique naturally transfers to node worker threads. It's unlikely to have some magical feature on Vitest out-of-the-box as noted in #3234 (comment)

   Regarding tsx loader failing on Worker constructor, this seems like a well known issue/limitation of current Node loader:

   • New --import API breaks worker threads privatenumber/tsx#354
   • ESM loaders cannot be defined via Worker option execArgv in v20 nodejs/node#47747

   The tsx issue suggests one workaround privatenumber/tsx#354 (comment) and that would probably work.

   👍React with 👍1Reacted by Andreas Huber
3. sheremet-va commented on May 22, 2024

   

   sheremet-va

   on May 22, 2024

   Member

   More actions

   The equivalent cannot work just fine in the browser mode because the browser mode doesn't support node:worker_threads module.

   If you are testing web worker, the recommendation is to use @vitest/web-worker package.
4. sheremet-va

   closed this as not plannedon May 22, 2024
5. andreashuber69 commented on May 22, 2024

   

   andreashuber69

   on May 22, 2024

   Author

   More actions

   The equivalent cannot work just fine in the browser mode because the browser mode doesn't support node:worker_threads module.

   With equivalent code I meant code implemented in terms of web worker, see test-browser script.
6. andreashuber69 commented on May 22, 2024

   

   andreashuber69

   on May 22, 2024

   Last edited by andreashuber69

   Author

   More actions

   For anyone landing here looking for a solution: I didn't get it to work. I tried with the following lines

           const tsx = new URL("../node_modules/tsx/dist/cli.mjs", import.meta.url);
           const worker = new Worker(tsx, { env: SHARE_ENV, argv: ["./src/nodeWorker.ts"] });
   

   Both cli.mjs and nodeWorker.ts are found, but apparently the message posted on the main thread never makes it to the worker.
7. hi-ogawa commented on May 23, 2024

   

   hi-ogawa

   on May 23, 2024

   Last edited by hi-ogawa

   Collaborator

   More actions

   Oh right, tsx probably spawns extra process in between, so Node's out-of-the-box worker communication doesn't work.

   Just reading back the original Node issue nodejs/node#47747 (comment), it might be that the fix is near in Node 22, but I just double checked it still doesn't work (and it actually breaks more for some use cases).

   For the workaround on Node 20, I think you can use tsx/esm/api like this to wrap your worker. I just tested this approach here https://github.com/hi-ogawa/reproductions/blob/main/vitest-5757-ts-worker/src/ts-worker/index.mjs

   import { Worker } from "node:worker_threads";
   
   class TsWorker extends Worker {
     constructor(filename, options = {}) {
       options.workerData ??= {};
       options.workerData.__ts_worker_filename = filename.toString();
       super(new URL("./worker.mjs", import.meta.url), options);
     }
   }
   
   const worker = new TsWorker(new URL("./your-actual-worker.ts", import.meta.url));

   // worker.mjs
   import { tsImport } from "tsx/esm/api";
   import { workerData } from "node:worker_threads";
   
   tsImport(workerData.__ts_worker_filename, import.meta.url);

   🎉React with 🎉4Reacted by Andreas Huber, Brad Christensen, Daniel Nagy and Dylan Moore
8. andreashuber69 commented on May 23, 2024

   

   andreashuber69

   on May 23, 2024

   Last edited by andreashuber69

   Author

   More actions

   For the workaround on Node 20, I think you can use tsx/esm/api like this to wrap your worker.

   Indeed, that works. I've modified vitest-repro accordingly and adapted the instructions for the reproduction to check out the original code.

   Thank you very much!
9. daniel-nagy commented on Jun 3, 2024

   

   daniel-nagy

   on Jun 3, 2024

   More actions

   If you don't want the stain of a non-typescript file on your filesystem (or just want to avoid the extra file)

   import { type WorkerOptions, Worker } from "node:worker_threads";
   
   const worker = /* JavaScript */ `
     import { createRequire } from "node:module";
     import { workerData } from "node:worker_threads";
   
     const filename = "${import.meta.url}";
     const require = createRequire(filename);
     const { tsImport } = require("tsx/esm/api");
     
     tsImport(workerData.__ts_worker_filename, filename);
   `;
   
   export class TsWorker extends Worker {
     constructor(filename: string | URL, options: WorkerOptions = {}) {
       options.workerData ??= {};
       options.workerData.__ts_worker_filename = filename.toString();
       super(new URL(`data:text/javascript,${worker}`), options);
     }
   }

   👍React with 👍2Reacted by Andreas Huber and Ricky Curtice
10. github-actions

    locked and limited conversation to collaborators on Jun 19, 2024