Python file with more than one entry breaks module resolution #3727

Activity

1. jaredcm commented on Dec 17, 2018

   

   jaredcm

   on Dec 17, 2018

   Last edited by jaredcm

   Author

   More actions

   I tried to create a brand new project and recreate this. The custom module is resolved without issue. When I try and import modules from my initial project I still cannot resolve them. In both projects I get an error message in the workload console like:

   /Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:303 [Extension Host] Python Extension: Failed to parse Prospector output SyntaxError: Unexpected end of JSON input
   	at JSON.parse (<anonymous>)
   	at module.exports.t.Prospector.<anonymous> (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:585133)
   	at Generator.next (<anonymous>)
   	at module.exports.r (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:584675)
   	at new Promise (<anonymous>)
   	at module.exports.r (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:584452)
   	at module.exports.t.Prospector.parseMessages (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:585083)
   	at module.exports.t.Prospector.<anonymous> (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:1:77469)
   	at Generator.next (<anonymous>)
   	at s (/Users/jared/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:1:75269)
   	at <anonymous>
   t.log @ /Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:303
   
2. jaredcm

   changed the title [-]Python linter not able to resolve paths specified in pth file.[/-] [+]Corrupted mypy cache prevents vscode from finding modules.[/+] on Dec 17, 2018
3. jaredcm commented on Dec 17, 2018

   

   jaredcm

   on Dec 17, 2018

   Last edited by jaredcm

   Author

   More actions

   Found the issue. Looks to be a corrupted .mypy_cache. I removed the cache directory from my initial project and the imports started working in both projects.
4. Brett Cannon (brettcannon)

   added

   triage

   on Dec 17, 2018
5. Brett Cannon (brettcannon)

   assigned

   Derek Keeler (d3r3kk)

   on Dec 17, 2018
6. jaredcm commented on Dec 18, 2018

   

   jaredcm

   on Dec 18, 2018

   Last edited by jaredcm

   Author

   More actions

   Looks like I was wrong. I restarted my computer and now my custom module cannot be resolved again. I remember I had been messing about with my pth file. Normally I have 4 paths in it which are all loaded into Python without issue. While filling out this bug report, I had removed all but 1, which was when VS Code started working. I had to add the other 3 paths back again later for my scripts to function properly. Which is why it stopped working and I didn't notice until VSCode was restarted.

   project.pth =
   os_path/project
   os_path/project/lib
   os_path/project/tests
   os_path/project/unit_tests

   I've done some testing:

   1. I added a project_lib.pth file which has 1 entry pointing to my lib directory and removed the entry from my project.pth file. The custom module is not resolvable.
   2. I change the name of project.pth to project.orig and restart VSCode. The module is resolvable.
   3. I change the name of project.orig back to project.pth and restart VSCode. The module is unresolvable again.
   4. I rename project.pth again, create a new file named project_root.pth, add the root directory for my project and restart VSCode. The module is resolvable as are modules in my project's root directory.

   It looks like VSCode gets confused when pth files have more than one entry.
7. jaredcm

   changed the title [-]Corrupted mypy cache prevents vscode from finding modules.[/-] [+]PTh file with more than one entry break module resolution[/+] on Dec 18, 2018
8. jaredcm commented on Dec 18, 2018

   

   jaredcm

   on Dec 18, 2018

   Last edited by jaredcm

   Author

   More actions

   New issue:
   In custom_module I have many sub_modules and they have other sub_modules. Within each of the sub_modules I use relative imports to load their child classes. The sub_module directory has a __init__,py with the import statement of "from .rest import Rest". This should resolve to a child directory named 'rest' that has a __init__.py file where the Rest class is defined. The full resolution from the root module to the imported class is 'custom_module.sub_module.rest.Rest'

   The error message is:
   {
   "resource": "~/project/lib/custom_module/sub_module/__init__.py",
   "owner": "generated_diagnostic_collection_name#0",
   "code": "unresolved-import",
   "severity": 4,
   "message": "unresolved import 'lib.custom_modulerest'",
   "source": "Python",
   "startLineNumber": 6,
   "startColumn": 6,
   "endLineNumber": 6,
   "endColumn": 11
   }

   Also notice that the message is not showing the correct module import path. The lib directory is not a module as it does not have a __init__.py file. If I change the path of the relative import to a full module import of 'from custom_module.sub_module.rest import Rest', then the error goes away. The path in the message is also missing the current module. It's almost like it the path it is looking for is shifted too far up the tree. Instead of looking in custom_module/sub_module/rest, I think it is looking for lib/custom_module/rest which doesn't exist.

   Developer Console output

     ERR Tree element not found: [object Object]: Error: Tree element not found: [object Object]
       at e.getElementLocation (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:214:266)
       at e.isCollapsed (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:213:757)
       at i.e.isCollapsed (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:962:723)
       at i.e.isExpanded (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:962:786)
       at t.revealMarkersForCurrentActiveEditor (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:4710:827)
       at t.autoReveal (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:4710:647)
       at t.onActiveEditorChanged (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:4707:826)
       at e.fire (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:189:532)
       at t.doEmitActiveEditorChangeEvent (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:4720:942)
       at t.handleActiveEditorChange (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:4720:718)
       at file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:4721:55
       at e.fire (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:189:512)
       at file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:5170:861
       at Object.g [as _notify] (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:167:807)
       at Object.enter (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:171:85)
       at n.Class.derive._creator._run (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:172:932)
       at n.Class.derive._creator._completed (file:///Applications/Design/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:172:373)
       at <anonymous>
       at process._tickCallback (internal/process/next_tick.js:188:7)
   console.ts:134 [Extension Host] Python Extension: Failed to parse Prospector output SyntaxError: Unexpected end of JSON input
   	at JSON.parse (<anonymous>)
   	at module.exports.t.Prospector.<anonymous> (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:585133)
   	at Generator.next (<anonymous>)
   	at module.exports.r (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:584675)
   	at new Promise (<anonymous>)
   	at module.exports.r (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:584452)
   	at module.exports.t.Prospector.parseMessages (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:83:585083)
   	at module.exports.t.Prospector.<anonymous> (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:1:77469)
   	at Generator.next (<anonymous>)
   	at s (/~/.vscode/extensions/ms-python.python-2018.12.1/out/client/extension.js:1:75269)
   	at <anonymous>
   t.log @ console.ts:134
   t._logExtensionHostMessage @ extensionHost.ts:453
   (anonymous) @ extensionHost.ts:244
   emitTwo @ events.js:126
   emit @ events.js:214
   emit @ internal/child_process.js:772
   _combinedTickCallback @ internal/process/next_tick.js:141
   _tickCallback @ internal/process/next_tick.js:180
   
9. Derek Keeler (d3r3kk)

   changed the title [-]PTh file with more than one entry break module resolution[/-] [+]Python file with more than one entry breaks module resolution[/+] on Jan 17, 2019
10. d3r3kk commented on Jan 17, 2019

    

    Derek Keeler (d3r3kk)

    on Jan 17, 2019

    Last edited by d3r3kk

    More actions

    Thanks for bringing this up jaredcm, an interesting problem!

    I didn't know about .pth files until this issue popped up so thank you for that.

    I can repro this issue with the latest stable VS Code (1.30.2) + latest extension (2018.12.1), using the Jedi language server ("python.jediEnabled": true").

    Workaround: The Microsoft Python Language Server (as of version 0.1.72.0, or greater) seems to handle this case (see below). Have you attempted to use this instead of Jedi? See this issue for details on how to enable/update the Microsoft Python Language Server.

    Adding more than one entry.

    Actually, on second look I have found that this is not handled properly by the Microsoft Python Language Server as I'd originally stated (at least version 0.1.72.0 doesn't) as I'd left out the multiple lines in the .pth part after I got things up and running.

    Indeed, if you add a .pth file to your site-packages with > 1 path, the issue does repro.

    Here's the sample workspace:

    3727_module_resolution.zip

    Here's the sample .pth file contents that I placed in the site-packages. I created a .venv in my workspace and copied this file to ${workspaceFolder}/.venv/Lib/site-packages/test_3727.pth.

    C:\\path\\to\\workspaceFolder\\3727_module_resolution
    C:\\path\\to\\workspaceFolder\\3727_module_resolution\\lib
    

    👍React with 👍1Reacted by jaredcm
11. Derek Keeler (d3r3kk)

    added

    bugIssue identified by VS Code Team member as probable bugIssue identified by VS Code Team member as probable bug

    area-intellisenseLSP-related functionality: auto-complete, docstrings, navigation, refactoring, etc.LSP-related functionality: auto-complete, docstrings, navigation, refactoring, etc.

    needs PR

    upstream-language server

    and removed

    triage

    needs PR

    on Jan 17, 2019
12. Derek Keeler (d3r3kk)

    removed their assignment

    on Jan 17, 2019
13. Brett Cannon (brettcannon)

    added

    needs upstream fix

    on Feb 19, 2019
14. Luciana Abud (luabud)

    added

    -review-

    on Sep 11, 2019
15. luabud commented on Sep 11, 2019

    

    Luciana Abud (luabud)

    on Sep 11, 2019

    Member

    More actions

    Closing in favour of microsoft/python-language-server#537. If you experience this with jedi, please file an issue on the jedi repo
16. Luciana Abud (luabud)

    closed this as completedon Sep 11, 2019
17. ghost removed

    needs upstream fix

    on Sep 11, 2019
18. lock

    locked as resolved and limited conversation to collaborators on Sep 19, 2019