Repository navigation
Test collector bug when --rootdir is not real path #9008
Description
- a detailed description of the bug or problem you are having
pytest test collector can get wrong item nodeid when running with a non --rootdir is not real path.
Actually I am studying a strange vscode-python extension behavior that when Linux workspace directory is not real path, vscode-python's test discovery gets wrong info about test file path. Tracking it down now I believe it's caused by this bug in pytest.
- output of
pip listfrom the virtual environment you are using
(.venv) $ pip list
Package Version
---------- -------
attrs 21.2.0
iniconfig 1.1.1
packaging 21.0
pip 21.2.4
pluggy 0.13.1
py 1.10.0
pyparsing 2.4.7
pytest 6.2.4
setuptools 41.6.0
toml 0.10.2
- pytest and operating system versions
pytest: 6.2.4 as shown above
OS: RHEL 8 (I believe other Linux can reproduce this too)
- minimal example if possible
Our demo directory structure (on Linux),
(.venv) $ tree
.
├── mypytest
└── tests
├── __init__.py
└── test_dummy.py
1 directory, 3 files
To see this issue we use a custom plugin to override pytest_collection_finish, to print the collected items nodeid to a log file.
(.venv) $ cat mypytest
#!/usr/bin/env python3
import sys
import pytest
class MyCollector:
def __init__(self, logfile):
self.logfile = logfile
def pytest_collection_finish(self, session):
with open(self.logfile, "w") as f:
for item in session.items:
f.write("%s\n" % dict(nodeid=item.nodeid, fspath=item.fspath))
if __name__ == "__main__":
plugin = MyCollector("./collector.log")
pytest.main(sys.argv[1:], [plugin])
Run test collection from the "tests" subdir: With --rootdir be the project root, if --rootdir is real path, result is good in that nodeid is 'tests/test_dummy.py' as relative to the rootdir. Buf if --rootdir is not real path, like if it's a symlink, the collected test has wrong nodeid.
# chdir to "tests"
(.venv) $ cd tests
(.venv) $ pwd
/var/tmp/pytest-issue/tests
# this is good
(.venv) $ ../mypytest --collect-only --rootdir /var/tmp/pytest-issue > /dev/null && cat collector.log
{'nodeid': 'tests/test_dummy.py::test_plus', 'fspath': local('/var/tmp/pytest-issue/tests/test_dummy.py')}
# bad if rootdir is not realpath: let's run from /var/tmp/foo which is not realpath.
# fspath is good but nodeid has not the "tests" subdir.
(.venv) $ ln -s /var/tmp/pytest-issue /var/tmp/foo
(.venv) $ cd /var/tmp/foo/tests
(.venv) $ ../mypytest --collect-only --rootdir /var/tmp/foo > /dev/null && cat collector.log
{'nodeid': 'test_dummy.py::test_plus', 'fspath': local('/var/tmp/pytest-issue/tests/test_dummy.py')}
vscode-python extension internally does something similar as above command for pytest test discovery. So when our workspace dir is not real path, we get problem with test discovery in vscode: while it detects the test files, it gets wrong idea about their path, causing individual run on a test file to fail.
Activity
kdkavanagh commentedon Feb 1, 2022
@stphnlyd curious if you ever found a solution to this? Hitting this as well...
azachar commentedon Nov 10, 2023
I've identified a workaround for the Python WSL debugging issue with symlinks in VSCode. By starting VSCode from the actual real location (not the symlinked one), test discover magically works again.
Upon further investigation, the issue seems linked to the Config class's
invocation_params.dirpytest/src/_pytest/main.py
Line 672 in 6fe4391
For instance, a symlink path like /my-symlink/test-folder pointing to /mnt/somewhereElse/test-folder leads to issues. The system resolves to "hidden" file location (/mnt/somewhereElse/test-folder), but it should be targeting the symlink (/my-symlink/test-folder).