c8 + vitest: Some suites get covered other suites ...dont?

I'm working on modernizing and open-sourcing some platform...
It's test strategy and test suite is built of two layers:

test/e2e - end-to-end suites that uses child_process.exec to run the SUT as cli in multiple scenarios.
test/unit - unit-level suites that add unit-tests to cover specific code-paths which are not covered by any of the e2e scenarios.

In the old setup I used nyc, but after modernizing it to recent version of node and refactoring from require to import I could not get nyc to produce coverage for the e2e suites. Then I found about c8.

Astonishingly, c8 produced coverage for the e2e right out of the box with zero configuration.
However, evidently, I was too fast to rejoice: It does not produce coverage for the unit tests... 🫨

The opened codebase is still young and does not take long to install or run - so I hope it can be used as a reproduction scenario...

Reproduction

Clone https://gitlab.com/prunjs/prun-core, use branch c8-bug-report.
install (npm i)
"vanilla" run - npm -w ecosystem/prun run cover
- Note that 3 suites have run: 1 e2e and 2 unit tests.
run only the e2e suites: npm -w ecosystem/prun run cover -- test/e2e
- expect: coverage numbers to be lesser than the vanilla run
- found: coverage are exactly like the "vanilla" run (i.e the "vanilla" run gathers coverage only from the e2e part)
run only unit tests - npm -w ecosystem/prun run cover -- test/unit
- expect: some coverage to be produced
- found: The only covered file is vitest.config.js... no coverage is reported for any of the project files

I believe I could do a mish-mash where I ask both tools to produce coverage report in json for the parts it covers successfully, and ask either of the tools to merge them, but that would be a patch. I thought to ask here before I embark on something weird like that...

I hope it's just something silly I'm missing... donno...

🙏 thanks in advance 🙏

Version:

10.1.2

Platform:
Reproduced in both
fedora in WSL2 , with node v20.17.0, npm `v10.8.2'
Win11 pro, with node v20.12.2, npm v10.5.0

@bcoe

@bcoe - I'd really appreciate it if you could take a look, or direct me to who can help

Thanks for all the good work! :)

You should use vitest coverage directly instead of c8 + vitest

vitest run --coverage

or c8 + mocha (or node test runner)

@osher

You should use vitest coverage directly instead of c8 + vitest

@osher did running vitest's built in coverage support do the trick for you? I recommend this, because it's one less layer of indirection.

@bcoe

Hi @bcoe,

I have exactly the same issue. With vitest's built in coverage, only my unit tests are covered, and with c8 only my integration tests (that spawn an HTTP server in a child process) are covered.

Do you have an idea on how to get the best of both worlds ?

Navigation Menu

Repository navigation

c8 + vitest: Some suites get covered other suites ...dont? #545