Repository navigation
Is pytest output guaranteed to contain test file paths to relative to cwd #7463
Description
For instance, in my case I had run the following command and got this output
C:\Users\karraj\AppData\Local\Programs\Python\Python37\python.exe c:\Users\karraj\Desktop\vscode-python\pythonFiles\pyvsc-run-isolated.py pytest --override-ini junit_family=xunit1 --rootdir c:\Users\karraj\Desktop\Desktop\test_demo --junit-xml=C:\Users\karraj\AppData\Local\Temp\tmp-41724BpAkWghx9dx.xml Tests/steve
================================== FAILURES ===================================
_______________________ FailingTests.test_failure_still _______________________
self = <hello_test.FailingTests testMethod=test_failure_still>
def test_failure_still(self):
> self.assertEqual("I'm right!", "no, I am!")
E AssertionError: "I'm right!" != 'no, I am!'
E - I'm right!
E + no, I am!
Tests\steve\hello_test.py:6: AssertionError
- generated xml file: C:\Users\karraj\AppData\Local\Temp\tmp-4172kPl1yrfyWOkV.xml -
============================== 1 failed in 0.12s ==============================
Tests\steve\hello_test.py is not the absolute path here, but the path relative to my current working directory. Is it guaranteed to contain relative paths always?
Activity
nicoddemus commentedon Jul 8, 2020
Hi @karrtikr,
Those paths are actually related to the
rootdir. You can see the rootdir in the header.The best way to make sure you have the same
rootdiralways is by creating apytest.iniin the location you want.karrtikr commentedon Jul 8, 2020
Ah I see, but it'll always be relative to the
rootdir, correct?Given this assumption, in case of windows, our extension had made the assumption that the relative path cannot contain
":"- do you think that's fair?There's some users who I suspect might be seeing absolute paths there. (microsoft/vscode-python#12403)
I will get back here as soon as I can once the user replies, mind keeping this open for a while?
nicoddemus commentedon Jul 8, 2020
I think it is possible, if the rootdir and the test file reside in different drives.
So you are one of the vscode developres? Cool! You guys are doing great work.
Btw, why are you interested in parsing the terminal output? I ask because for integration usually the best approach are hooks instead.
karrtikr commentedon Jul 8, 2020
Thanks, Python extension VSCode dev here 👋
That's fair. But for our extension, we force the rootdir to be the workspace root, and only support running test files inside the workspace root, so they should be on the same drive.
I am not entirely sure, we were parsing output long before I joined the team. Maybe because hooks were not there earlier, or were not sufficient? I can consult with my team on whether we want to move towards this.
karrtikr commentedon Jul 8, 2020
@nicoddemus Can you please point to these hooks which we can use to run pytest where we won't need to parse the result?
nicoddemus commentedon Jul 8, 2020
Cool 👋
Oh I see. I remember @brettcannon tweeting something along the lines of parsing the output as an early implementation, but that is brittle because we don't treat our output as stable, but he was aware that hooks were the way to go.
They were introduced quite a long time ago, in 2010. I believe they should be more than enough to get any information you need from a pytest run, and they are quite stable.
A quick example on how vscode could integrate with pytest using hooks:
The list of hooks can be found in the API Reference, but I bet the more interesting ones for VS integration would be the reporting hooks.
Please feel free to ask any questions, we are more than glad to help with getting the best pytest integration possible in vscode!
brettcannon commentedon Jul 14, 2020
So we use the hooks for discovery, but we don't for execution (yet; we plan to, but no clue on an ETA).
Zac-HD commentedon Aug 3, 2020
Hey all 👋
It looks like we've answered this question so I'm closing the issue; feel free to open another if you have more questions.