☂️ [Roadmap]: Improve Testing #2712

This is an umbrella issue to related to one of our priorities for code-server: testing.

There are three main areas we're focusing on in regards to code-server's testing situation.

1. Code Coverage

We need to be able to measure our testing efforts. The best way to do that is through code coverage.

• ✅ Add support for code coverage
  • refactor(tests): migrate from mocha to jest #2564
  • feat(testing): add code coverage #2621
• ✅ Track code coverage for Lines in README feat(testing): add code coverage #2621
• 🚧 Increase Lines coverage to 90-100% (currently: 50.26%)
  • feat(testing): add unit tests for common/util #2671
  • feat(testing): add unit tests for constants #2701
  • feat(testing): add unit tests for src/common/http #2702
  • feat(testing): add unit tests for emitter #2715
  • feat(testing): add unit tests for register #2719
  • feat(testing): add tests for serviceWorker.ts #2744
• ⏳ Increase threshold to 90% and fail CI if not met
• ⏳ Track code coverage with each PR using something like @codecov

2. End-to-End Tests

code-server is a complex app and deserves to be thoroughly tested. With end-to-end tests, we can fix bugs, prevent regressions and ship more confidently.

• ✅ Add support for end-to-end tests feat(testing): add playwright  #2643
• ✅ Add end-to-end test for "Go Home" feature feat(testing): add e2e test for 'Go Home' button #2648 (removed in feat(vscode): update to version 1.54.2 #2862)
• ⏳ Add end-to-end test for installing extension from VSIX file
• ⏳ Add end-to-end test for using the Integrated Terminal
• ⏳ Add end-to-end test for using the File Search
• TODO - will add more here

3. Test Guidelines

To make it easier for both contributors and maintainers to write tests for code-server, we should have guidelines.

• ⏳ Add /docs/testing.md with guidelines for writing unit, integration and end-to-end tests

Related issues:

• [dev]: separate unit and end-to-end testing #2646

LEGEND

⏳ = to do
🚧 = in progress
✅ = done

Note: this is part of an experiment to be more transparent about our priorities and our roadmap. See everything related to this effort in the project board