[Request] Hide the JUnit Callstack from the stacktrace

There are (few, but there are) cases in which I'd like to take a look at the callstack for a failed (as opposed errored) test. Mainly when using assertThrows. However I would like to have it shortened by default so that in the simplest case every line starting with at org.junit is replaced with the ... X more lines just like the end is shortened right now.

I'd be satisfied with a setting like "shorten lines matching this regex from the stack trace".

For example when assertThrows fails, because instead of ExpectedException I got DifferentException, I'd love the output to look like

org.opentest4j.AssertionFailedError: Unexpected exception type thrown ==> expected: <org.example.ExpectedException> but was: <org.example.DifferentException>
    at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:65)
    ... 70 more lines
Caused by: org.example.DifferentException: Message
    at x
    at y
    at z

Leona Gottfried (@leogott)

Leona Gottfried (@leogott) Could you give an example of what's the original stack trace looks like?

In the example you gave, why we just hide the first part of the stack trace but leave the second one as it is?

For a small example I wrote (junit5 assert throws expects A but receives B caused by C), I get this unabbreviated result.

Every line beginning with at org.junit.** (for example) is part of the testing implementation and I don't care about the junit internals, because I'm confident my own code is the likely cause of the failed test.

I don't care about the callstack of the AssertionFailedError (beyond that it was in line 25 of AppTest.java) , because it was thrown by the assertThrows method.

I do however care about the Cause (the exception of wrong type), and where it originated.

So in this case I'm getting the following information:

%TESTC  1 v2
%TSTTREE2,org.example.AppTest,true,1,false,1,AppTest,,[engine:junit-jupiter]/[class:org.example.AppTest]
%TSTTREE3,someFeature_stackTrace(org.example.AppTest),false,1,false,2,someFeature_stackTrace(),,[engine:junit-jupiter]/[class:org.example.AppTest]/[method:someFeature_stackTrace()]
%TESTS  3,someFeature_stackTrace(org.example.AppTest)
%FAILED 3,someFeature_stackTrace(org.example.AppTest)
%TRACES 
org.opentest4j.AssertionFailedError: Unexpected exception type thrown ==> expected: <java.io.IOException> but was: <java.lang.IllegalArgumentException>
        at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:65)
        at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:37)
        at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3007)
        at org.example.AppTest.someFeature_stackTrace(AppTest.java:25)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
        at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
        at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
        at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
        at java.util.ArrayList.forEach(ArrayList.java:1259)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
        at java.util.ArrayList.forEach(ArrayList.java:1259)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:84)
        at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98)
        at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: java.lang.IllegalArgumentException: Argument must not be null
        at org.example.App.someFeatureImTesting(App.java:20)
        at org.example.AppTest.lambda$0(AppTest.java:26)
        at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:55)
        ... 70 more
Caused by: java.lang.NullPointerException
        at org.example.App.someFeatureImTesting(App.java:17)
        ... 72 more
%TRACEE 
%TESTE  3,someFeature_stackTrace(org.example.AppTest)
%RUNTIME175


> Test run finished at 21.8.2021, 01:23:53 <

But this is what I care about:

TEST: someFeature_stackTrace(org.example.AppTest)
RESULT: failed
TRACE:
org.opentest4j.AssertionFailedError: Unexpected exception type thrown ==> expected: <java.io.IOException> but was: <java.lang.IllegalArgumentException>
        ... 3 more
        at org.example.AppTest.someFeature_stackTrace(AppTest.java:25)
        ... X more
Caused by: java.lang.IllegalArgumentException: Argument must not be null
        at org.example.App.someFeatureImTesting(App.java:20)
        at org.example.AppTest.lambda$0(AppTest.java:26)
        ... 71 more
Caused by: java.lang.NullPointerException
        at org.example.App.someFeatureImTesting(App.java:17)
        ... 72 more

Would it be possible to at least cut the callstack for the AssertionFailed Exception that fails the Test?

I see, so that is to only reserve the stack trace which belongs to the source?

Oh you're correct. I didn't look at it that way, but it absolutely is just the parts of the callstack within the source at test.

Yes, I'd like to see a filtered view or something that cuts away the callstack that doesn't belong to the source.

Get a very similar feedback from a customer interview. He hope the stacktrace can be hidden by clicking some buttons like show more, show less

Leona Gottfried (@leogott)

Hi Leona Gottfried (@leogott),

I pushed a change to filter out those 'internal' traces. Currently we don't have a setting to control it. But we can definitely add a setting for that if users require.

You can try the new behavior in the next pre-release of the extension (should be available tomorrow).

Let me know if you have any feedback, thanks!

Jinbo Wang (@testforstephen)

To verify this issue

Run a JUnit test case which will fail.
Check the stacktrace, it should not contain following packages/classes:
- org.eclipse.jdt.internal.junit.runner.*
- org.eclipse.jdt.internal.junit4.runner.*
- org.eclipse.jdt.internal.junit5.runner.*
- org.eclipse.jdt.internal.junit.ui.*
- junit.framework.TestCase
- junit.framework.TestResult
- junit.framework.TestResult$1
- junit.framework.TestSuite
- junit.framework.Assert
- org.junit.*
- java.lang.reflect.Method.invoke
- sun.reflect.*
- jdk.internal.reflect.*

// cc Jinbo Wang (@testforstephen)

The latest implementation works well. One minor suggestion is to hide the highlighting two call stacks as well, since they doesn't provide any valid info for diagnose.

Navigation Menu

Repository navigation

[Request] Hide the JUnit Callstack from the stacktrace #1281

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Request] Hide the JUnit Callstack from the stacktrace #1281

Description

Activity

jdneo commented on Aug 19, 2021

leogott commented on Aug 21, 2021

jdneo commented on Aug 21, 2021

leogott commented on Aug 21, 2021

jdneo commented on Nov 25, 2021

jdneo commented on Apr 7, 2022

jdneo commented on Apr 22, 2022

testforstephen commented on Apr 26, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions