Error: spawn <python interpreter> ENOENT on test discovery when "python.testing.cwd" is set

I have searched stackoverflow as well as GitHub issues here and found some related issues but nothing conclusive or helpful.

Environment data

VS Code version: 1.54.3
Extension version (available under the Extensions sidebar): v2021.3.658691958
OS and version: macOS Big Sur 11.2.3
Python version (& distribution if applicable, e.g. Anaconda): all
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): n/a
Relevant/affected Python packages and their versions: pytest | unittest
Relevant/affected Python-related VS Code extensions and their versions: vscode-python v2021.3.658691958
Value of the python.languageServer setting: pylance

Expected behaviour

python process for test discovery would spawn successfully or produce more descriptive error.

Actual behaviour

python /Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /Users/patrick/dev/vscode-test -s --cache-clear
Test Discovery failed: 
Error: spawn /Users/patrick/.pyenv/shims/python ENOENT

Steps to reproduce:

Delete/move ~/.vscode for fresh vscode user env.
Create simple python project with one pytest test and open vscode to folder with only the python extension and pytest auto discovery enabled.
Observe output in Python Test Log listed below.
Copy/paste command line from Python Test Log into macOS terminal.
Observe process runs successfully and discovers pytest test.

*NOTE: I am unable to reproduce this problem on a clean system. But, it occurs for any python 3 version installed via system, brew, and pyenv regardless of whether the vscode project points to a symlink or executable.

Logs

Output for Python in the Output panel

User belongs to experiment group 'pythonaacf'
User belongs to experiment group 'pythonSendEntireLineToREPL'
User belongs to experiment group 'pythonNotDisplayLinterPrompt'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'ShowExtensionSurveyPrompt - enabled'
User belongs to experiment group 'CollectLSRequestTiming - control'
> conda --version
> conda info --json
> pyenv root
> python3.7 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3.6 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python2 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3 ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> pyenv root
> pyenv root
> pyenv root
> pyenv root
> pyenv root
> pyenv root
Starting Pylance language server.
Python interpreter path: ~/.pyenv/shims/python
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/dev/vscode-test -s --cache-clear
cwd: api
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/dev/vscode-test -s --cache-clear
cwd: api
Error 2021-03-20 20:11:59: Failed to parse discovered Test [Error: spawn /Users/patrick/.pyenv/shims/python ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)] {
  errno: 'ENOENT',
  code: 'ENOENT',
  syscall: 'spawn /Users/patrick/.pyenv/shims/python',
  path: '/Users/patrick/.pyenv/shims/python',
  spawnargs: [
    '/Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py',
    'discover',
    'pytest',
    '--',
    '--rootdir',
    '/Users/patrick/dev/vscode-test',
    '-s',
    '--cache-clear'
  ]
}
> conda --version
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import pytest"
> ~/.pyenv/shims/python ~/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/pyvsc-run-isolated.py -c "import pytest"
Error 2021-03-20 20:12:00: Python Extension: displayDiscoverStatus [TypeError: Cannot read property 'uri' of undefined
	at /Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/out/client/extension.js:9:495273]

Output for Python Test Log in the Output panel

python /Users/patrick/.vscode/extensions/ms-python.python-2021.3.658691958/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /Users/patrick/dev/vscode-test -s --cache-clear
Test Discovery failed: 
Error: spawn /Users/patrick/.pyenv/shims/python ENOENT

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority.

Thanks for your understanding and patience!

Dr. Patrick Stinson (@patrickkidd)

Hi Dr. Patrick Stinson (@patrickkidd) ,

It seems you're using the pyenv shims as your Python interpreter, /Users/patrick/.pyenv/shims/python. As a workaround, can you try pointing it to the actually python itself? It should probably be inside /Users/patrick/.pyenv/versions.

Also, please verify that the shim exists. ENOET is generally thrown when the command doesn't exist. (see this). Please try running /Users/patrick/.pyenv/shims/python in terminal.

Dr. Patrick Stinson (@patrickkidd)

Hi Dr. Patrick Stinson (@patrickkidd) ,

It seems you're using the pyenv shims as your Python interpreter, /Users/patrick/.pyenv/shims/python. As a workaround, can you try pointing it to the actually python itself? It should probably be inside /Users/patrick/.pyenv/versions.

Also, please verify that the shim exists. ENOET is generally thrown when the command doesn't exist. (see this). Please try running /Users/patrick/.pyenv/shims/python in terminal.

I appreciate your diligence with this issue. Yes. I saw the same potential problems with shims and symlinks in my own troubleshooting and have have no luck trying with multiple interpreters installed from system, brew, and pyenv, pointing to and not pointing to symlinks.

I did try a fresh user and pyenv install on the same machine and the plugin worked. So then for my original user I cleared out my .bash_profile, pyenv, and .vscode user settings are extensions and could not get it to work. Couldn't think of anything else that would effect the plugin and/or python environment. So I got stumped there.

I have a little extra info for this issue. This error goes away if I remove this line: from my workspace settings:

"python.testing.cwd": "api",

The "api" dir in my project repo root is where all the python is. Unfortunately I still need to set the cwd for pytest so now I have other errors related to that to deal with.

Thoughts?

I see, can you try running /Users/patrick/.pyenv/shims/python inside the api directory in terminal and see if the shim reports the same ENOET error?

It does not. All of the shims run correctly.

I was talking with my team and it was suggested that shims needed a shell to run on. In many parts of the extension like discovery, we use child_process.spawn which does not use a shell by default (unless shell: true option is specified), and hence this fails.

So I think we do not support shims as of now.

You know, it isn't working in any way now, even with "python.testing.cwd" set to "${workspaceFolder}/api" or removed entirely. I tried in two different projects.

Is there any way for me to debug the code calling spawn? Even if editing code and printing to console?

If I take the test discovery command from the Python Test Log output and run it manually with the same python interpreter set in the workspace, that command does succeed (though of course it has to succeed in VSCode so the extension can parse the output):

turin:familydiagram patrick$ python /Users/patrick/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /Users/patrick/dev/familydiagram -s --cache-clear -svv tests
[{"rootid": ".", "root": "/Users/patrick/dev/familydiagram", "parents": [{"id": "./tests", "kind": "folder", "name": "tests", "parentid": ".", "relpath": "./tests"}, {"id": "./tests/test_AccountDialog.py", "kind": "file", "name": "test_AccountDialog.py", "parentid": "./tests", "relpath": "./tests/test_AccountDialog.py"}, {"id": "./tests/test_DatePicker.py", "kind": "file", "name": "test_DatePicker.py", "parentid": "./tests", "relpath": "./tests/test_DatePicker.py"}, {"id": "./tests/test_TimelineSearch.py", "kind": "file", "name": "test_TimelineSearch.py", "parentid": "./tests", "relpath": "./tests/test_TimelineSearch.py"}, {"id": 

...

So there is something going on with that command when VSCode executes it that is causing ENOENT

Note you'll need to Reload for changes in the setting to take affect, the cwd in python output panel doesn't change unless we reload. Does removing the setting and reloading make a difference? For me it does fix the spawn enoet issue.

Also, you'll need to run that command in the cwd chosen when running in terminal. I'll find out tomorrow about the exact location where extension runs the command.

It looks like the Python section of the Output tab is printing some info that might be useful. Notice the spawn args output. I also switched to the binary of the system python 3.8 just to simplify things.

User belongs to experiment group 'pythonaacf'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'pythonSurveyNotificationcf'
User belongs to experiment group 'PythonPyTorchProfiler'
User belongs to experiment group 'ShowExtensionSurveyPrompt - control'
User belongs to experiment group 'CollectLSRequestTiming - control'
Python interpreter path: /usr/bin/python3
Starting Pylance language server.
> /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/dev/familydiagram -s --cache-clear -svv tests
cwd: ${workspaceDir}
> /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/dev/familydiagram -s --cache-clear -svv tests
cwd: ${workspaceDir}
Error 2021-05-24 22:52:28: Failed to parse discovered Test [Error: spawn /usr/bin/python3 ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
	at onErrorNT (internal/child_process.js:465:16)
	at processTicksAndRejections (internal/process/task_queues.js:80:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /usr/bin/python3',
  path: '/usr/bin/python3',
  spawnargs: [
    '/Users/patrick/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/testing_tools/run_adapter.py',
    'discover',
    'pytest',
    '--',
    '--rootdir',
    '/Users/patrick/dev/familydiagram',
    '-s',
    '--cache-clear',
    '-svv',
    'tests'
  ]
}
> /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/pyvsc-run-isolated.py -c "import pytest"
> /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/pyvsc-run-isolated.py -c "import pytest"
> conda --version
Error 2021-05-24 22:52:28: Python Extension: displayDiscoverStatus [TypeError: Cannot read property 'uri' of undefined
	at /Users/patrick/.vscode/extensions/ms-python.python-2021.5.842923320/out/client/extension.js:9:500976
	at runMicrotasks (<anonymous>)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)]

Looking at the logs, cwd is ${workspaceDir}, which does not seem right.

cwd: ${workspaceDir}

For me when done correctly with absolute path, it resolves to the full path. For eg. if I set "python.testing.cwd": "${workspaceFolder}\\tests" and reload window, then rediscover tests, the output panel states cwd as

cwd: c:\Users\karraj\OneDrive - Microsoft\Desktop\folders\pytestExample\tests

Please double check the value of python.testing.cwd you're using when creating the logs, and make sure it's pointing to a valid directory.

Note you'll need to Reload for changes in the setting to take affect, the cwd in python output panel doesn't change unless we reload. Does removing the setting and reloading make a difference? For me it does fix the spawn enoet issue.

Ah, that's right. Reloading the window did fix the spawn problem on both projects.

I have no idea where ${workspaceDir} came from, it isn't anywhere in my project folder or the extensions folder. Though it did appear when I removed the "python.testing.cwd" from my settings. It looks like the extension always prints cwd <whatver-it-is-set-to> on a new line.

And - setting it to ${workspaceFolder}/api did fix the cwd problem! Good grief....cheers for the help!

I don't want to hijack this issue but I think this is related. While the test list is loading fine, I am getting this when running a single test:
ERROR: file or directory not found: ERROR: file or directory not found: ./api/tests/modules/calsync/unit/test_model.py::test_create_calsync_record

I get the same result regardless of whether I click "Debug Test" within the source file or in the side bar, and regardless of whether "python.testing.cwd" is set, and whether it has an absolute path or not. It looks like

The command pytest -svvx ./api/tests/modules/calsync/unit/test_model.py::test_create_calsync_record does run and passes the test when run manually in the terminal from the workspace root.

The full command that the VSCode extension is using does run and succeed in the terminal as well:

~/dev/teamworks/.direnv/python-3.7.5/bin/python ~/.vscode/extensions/ms-python.python-2021.5.842923320/pythonFiles/pyvsc-run-isolated.py pytest --override-ini junit_family=xunit1 --rootdir ~/dev/teamworks --junit-xml=/var/folders/8t/zl93d_hj52x1jc35822119jc0000gn/T/tmp-20441zwX7fAIHmN8B.xml ./api/tests/modules/calsync/unit/test_model.py::test_create_calsync_record

I wonder if the VSCode extension is correctly setting the cwd to "${workspaceFolder}/api", and then it is forcing the full path of the test script to be from the workspace root instead of the cwd? Because I noticed that the test script path in the command line still has ./api prepended to it. So it would make sense that it can't find ./api/api/tests/modules/calsync/unit/test_model.py?

Glad you were able to fix discovery!

Actually debugging doesn't use python.testing.cwd setting, it uses the launch json schema. (Your issue seems similar to #11587 (comment)) Can you add following configuration to your launch.json file,

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug Unit Test",
            "type": "python",
            "request": "test",
            "cwd": "${workspaceFolder}\\api",
        }
    ]
}

and see if it helps.

Anyways this would be a different issue, do you mind opening a new one in case if you still face issues with it? Please provide all of the Python, Python Test log, and debug console logs as it really helps, thanks.

I have mentioned in #11669 that we should update the docs for python.testing.cwd, that it should only carry absolute paths. As it seems you were trying to use relative paths instead which failed. Closing in favor of #11669.

Navigation Menu

Repository navigation

Error: spawn <python interpreter> ENOENT on test discovery when `"python.testing.cwd"` is set #15739

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Error: spawn <python interpreter> ENOENT on test discovery when "python.testing.cwd" is set #15739

Description

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

Activity

karrtikr commented on Mar 22, 2021

karrtikr commented on Mar 29, 2021

patrickkidd commented on Mar 29, 2021

patrickkidd commented on Apr 8, 2021

karrtikr commented on Apr 8, 2021

patrickkidd commented on Apr 8, 2021

karrtikr commented on Apr 8, 2021

10 remaining items

patrickkidd commented on May 25, 2021

patrickkidd commented on May 25, 2021

karrtikr commented on May 25, 2021

karrtikr commented on May 25, 2021

patrickkidd commented on May 25, 2021

karrtikr commented on May 25, 2021

patrickkidd commented on May 25, 2021

patrickkidd commented on May 25, 2021

karrtikr commented on May 25, 2021

karrtikr commented on May 25, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Error: spawn <python interpreter> ENOENT on test discovery when `"python.testing.cwd"` is set #15739