Skip to content

Support pytest plugins #16852

Closed
Closed
Support pytest plugins#16852
Assignees
karthiknadig
Labels
area-testingfeature-requestRequest for new features or functionality

Description

@dcavanagh
Dan Cavanagh (dcavanagh)
opened on Aug 3, 2021

Environment data

  • VS Code version: 1.59.0-insider
  • Extension version (available under the Extensions sidebar): v2021.7.1060902895
  • OS and version: macOS 10.15.7
  • Python version (& distribution if applicable, e.g. Anaconda): 3.8.3
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pipenv
  • Value of the python.languageServer setting: Default

Expected behaviour

It discovers tests

Actual behaviour

The Python extension has run into an unexpected situation
while processing a pytest node during test discovery. Please
Please open an issue at:
https://github.com/microsoft/vscode-python/issues
and paste the following output there.

nodeid: test/views/test_stations.py::describe_get_stations_with_count::describe_loggernet_data_platform::data_platform::it_queries_station_count
kind: ('function', False)
class: Function
name: data_platform::it_queries_station_count
fspath: /Users/dan/groundwork/GroundNetEngine/test/views/test_stations.py
location: ('test/views/test_stations.py', 42, 'describe_get_stations_with_count.describe_loggernet_data_platform.data_platform::it_queries_station_count')
function: <function describe_get_stations_with_count..data_platform..it_queries_station_count at 0x106ad3d30>
markers: []
user_properties: []
attrnames: ['_ALLOW_MARKERS', 'annotations', 'class', 'delattr', 'dict', 'dir', 'doc', 'eq', 'format', 'ge', 'getattribute', 'gt', 'hash', 'init', 'init_subclass', 'le', 'lt', 'module', 'ne', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'slots', 'str', 'subclasshook', 'weakref', '_fixtureinfo', '_getobj', '_initrequest', '_nodeid', '_obj', '_prunetraceback', '_pyfuncitem', '_report_sections', '_repr_failure_py', '_request', '_store', 'add_marker', 'add_report_section', 'addfinalizer', 'cls', 'config', 'extra_keyword_matches', 'fixturenames', 'from_parent', 'fspath', 'funcargs', 'function', 'get_closest_marker', 'getmodpath', 'getparent', 'ihook', 'instance', 'iter_markers', 'iter_markers_with_node', 'keywords', 'listchain', 'listextrakeywords', 'listnames', 'location', 'module', 'name', 'nextitem', 'nodeid', 'obj', 'originalname', 'own_markers', 'parent', 'reportinfo', 'repr_failure', 'runtest', 'session', 'setup', 'teardown', 'user_properties', 'warn']

extra info:
fullname: describe_get_stations_with_count.describe_loggernet_data_platform.data_platform::it_queries_station_count
testfunc: describe_get_stations_with_count.describe_loggernet_data_platform.data_platform.it_queries_station_count
parameterized:

traceback:
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/run_adapter.py", line 22, in
main(tool, cmd, subargs, toolargs)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/main.py", line 100, in main
parents, result = run(toolargs, **subargs)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 30, in discover
ec = _pytest_main(pytestargs, [_plugin])
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/config/init.py", line 162, in main
ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 84, in
self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
return wrap_session(config, _main)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
session.exitstatus = doit(config, session) or 0
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 322, in _main
config.hook.pytest_collection(session=session)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 84, in
self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 333, in pytest_collection
session.perform_collect()
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 641, in perform_collect
hook.pytest_collection_finish(session=self)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 84, in
self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 107, in pytest_collection_finish
test, parents = self.parse_item(item)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
return parse_item(item)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 172, in parse_item
raise should_never_reach_here(
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 132, in should_never_reach_here
traceback.print_stack()
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 637, in perform_collect
INTERNALERROR> hook.pytest_collection_modifyitems(
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in call
INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 84, in
INTERNALERROR> self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 92, in pytest_collection_modifyitems
INTERNALERROR> test, parents = self.parse_item(item)
INTERNALERROR> File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
INTERNALERROR> return parse_item(item)
INTERNALERROR> File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 172, in parse_item
INTERNALERROR> raise should_never_reach_here(
INTERNALERROR> NotImplementedError: Unexpected pytest node (see printed output).
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR> config.hook.pytest_collection(session=session)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in call
INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 84, in
INTERNALERROR> self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 333, in pytest_collection
INTERNALERROR> session.perform_collect()
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/_pytest/main.py", line 641, in perform_collect
INTERNALERROR> hook.pytest_collection_finish(session=self)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in call
INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/manager.py", line 84, in
INTERNALERROR> self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/Users/dan/.local/share/virtualenvs/GroundNetEngine-WLpcZVa5/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 107, in pytest_collection_finish
INTERNALERROR> test, parents = self.parse_item(item)
INTERNALERROR> File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
INTERNALERROR> return parse_item(item)
INTERNALERROR> File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 172, in parse_item
INTERNALERROR> raise should_never_reach_here(
INTERNALERROR> NotImplementedError: Unexpected pytest node (see printed output).

========================= 22 tests collected in 4.56s ==========================

Traceback (most recent call last):
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/run_adapter.py", line 22, in
main(tool, cmd, subargs, toolargs)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/main.py", line 100, in main
parents, result = run(toolargs, **subargs)
File "/Users/dan/.vscode-insiders/extensions/ms-python.python-2021.7.1060902895/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 3)

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

Activity

  1. Dan Cavanagh (dcavanagh)
    added
    triage-neededNeeds assignment to the proper sub-team
    bugIssue identified by VS Code Team member as probable bug
    on Aug 3, 2021

  2. dcavanagh commented on Aug 3, 2021

    @dcavanagh
    Dan Cavanagh (dcavanagh)
    on Aug 3, 2021
    Author

    It fails to discover tests even if I delete all the test files except for one.

  3. karthiknadig commented on Aug 3, 2021

    @karthiknadig
    Karthik Nadig (karthiknadig)
    on Aug 3, 2021
    Member

    Dan Cavanagh (@dcavanagh) Can you share the function and its params? also are you using any pytest plugins?

  4. dcavanagh commented on Aug 3, 2021

    @dcavanagh
    Dan Cavanagh (dcavanagh)
    on Aug 3, 2021 via email
    Author
  5. Karthik Nadig (karthiknadig)
    self-assigned this
    on Aug 3, 2021
  6. Karthik Nadig (karthiknadig)
    added
    area-testing
    triage
    and removed
    triage-neededNeeds assignment to the proper sub-team
    on Aug 3, 2021
  7. Karthik Nadig (karthiknadig)
    changed the title [-]Test discovery failure[/-] [+]Support pytest plugins[/+] on Aug 4, 2021
  8. Karthik Nadig (karthiknadig)
    added
    needs PR
    feature-requestRequest for new features or functionality
    and removed
    triage
    bugIssue identified by VS Code Team member as probable bug
    on Aug 4, 2021

  9. karthiknadig commented on Aug 4, 2021

    @karthiknadig
    Karthik Nadig (karthiknadig)
    on Aug 4, 2021
    Member

    Our current discovery code does not support plugins like pytest-describe. We have plans to improve our discovery to handle plugins.

  10. dcavanagh commented on Aug 5, 2021

    @dcavanagh
    Dan Cavanagh (dcavanagh)
    on Aug 5, 2021
    Author

    Karthik Nadig (@karthiknadig) This particular plugin had been working and then stopped recently.

  11. roeap commented on Nov 24, 2021

    @roeap
    Robert Pack (roeap)
    on Nov 24, 2021

    Plugins like pytest-describe really help us in managing tests in complex code bases. At the same time some of our devs would like to use the testing UIs in vscode. Karthik Nadig (@karthiknadig) can you share any info if and when support for plugins might be worked on?

  12. karthiknadig commented on Nov 24, 2021

    @karthiknadig
    Karthik Nadig (karthiknadig)
    on Nov 24, 2021
    Member

    Robert Pack (@roeap) We are currently working on this. #17242 an objective with the re-write of the test adapter is to allow it to work with other plugins.

  13. roeap commented on Nov 24, 2021

    @roeap
    Robert Pack (roeap)
    on Nov 24, 2021

    Thanks for the update and this great extension. Looking forward for this to land.

  14. kr-hansen commented on Feb 22, 2022

    @kr-hansen
    Kyle Hansen (kr-hansen)
    on Feb 22, 2022

    Will note that in the meantime, this same problem can be solved by pinning pytest-describe==2.0.0 in the environment for testing. For some reason VSCode only breaks with pytest-describe==2.0.1

  15. github-actions
    removed
    needs PR
    on Aug 9, 2022
  16. Kartik Raj (karrtikr)
    added
    needs PRReady to be worked on
    on Aug 9, 2022

  17. illia-karpenkov commented on Aug 10, 2022

    @illia-karpenkov
    Illia (illia-karpenkov)
    on Aug 10, 2022

    Possibly related: #19668

  18. Mirandatz commented on Aug 26, 2022

    @Mirandatz
    Mirandatz (Mirandatz)
    on Aug 26, 2022

    Thanks to Kyle Hansen (@kr-hansen) comment, I found that uninstalling pytest-pspec "fixes" the issue.
    Thank you :D

  19. eleanorjboyd commented on Jul 14, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 14, 2023

    Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To try it yourself, you need to be on the most recent release of the Python extension and add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

    You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your user settings then check for Experiment 'pythonTestAdapter' is active in your python logs.

    Let me know if the rewrite fixes your issue. Thanks!

  20. eleanorjboyd commented on Oct 30, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Oct 30, 2023

    Generally plugins should now work on the rewrite, please open issues with specific plugins if this is not the case for you. Thanks

    Trying the rewrite:
    To use the rewrite yourself just add ”python.experiments.optInto": ["pythonTestAdapter"] to your user settings. You can confirm you have the rewrite enabled by setting your log level to trace, via the Developer: Set Log Level command in the command palette. Then check to see if Experiment 'pythonTestAdapter' is active is in your python logs.

  21. Eleanor Boyd (eleanorjboyd)
    closed this as completedon Oct 30, 2023
  22. github-actions
    removed
    needs PRReady to be worked on
    on Oct 30, 2023
  23. github-actions
    locked as resolved and limited conversation to collaborators on Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-testingfeature-requestRequest for new features or functionality

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions