Import JUnit Tests problems

#1

Hi there,

I stated in another topic that we try to import Google Test XML output with the JUnit import action in continua.
I got the new version (1.9.0.430) and that works great to use the file name as assembly value. But I found some other issues while looking at our test results.

First, if we don’t specify a namespace (just the classname) for every testcase, continua has problems with the test result table view. As you can see in the image below, only the one test with a specified ‘Namespace’ has the correct layout (all other tests have a Fixture name specified which is not displayed).

ContinuaCI_UnitTest_TableViewBug
ContinuaCI_UnitTest_TableViewBug.png1599×455 17.2 KB

Another issue we’ve experienced is with disabled tests (not shelved). We tried different ways, used disabled or skipped attribute, but all tests are always displayed as ‘Success’. The UnitTest section in the build “Details” page also doesn’t show a value for “disabled” tests (only shelved), which leads me to believe that continua doesn’t support disabled tests. Is our import test behavior working as expected for continua?

Used xml:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="4" failures="1" disabled="1" errors="0" timestamp="2019-05-08T10:09:24" time="7.171" name="AllTests">
  <testsuite name="Disabled.Tests1" tests="1" failures="0" skipped="1" disabled="1" errors="0" time="0.296">
    <testcase status="notrun" name="DISABLED_Test1_NotRun_Skipped" time="0" classname="Disabled.Tests1">
      <skipped message="TestSkipped1"><![CDATA[Disabled.Tests1]]></skipped>
    </testcase>
  </testsuite>
  <testsuite name="Disabled.Tests2" tests="1" failures="0" skipped="1" disabled="1" errors="0" time="0.296">
    <testcase status="notrun" name="DISABLED_Test2_NotRun_Disabled" time="0" classname="Disabled.Tests2">
      <skipped message="TestSkipped2"><![CDATA[Disabled.Tests2]]></skipped>
    </testcase>
  </testsuite>
  <testsuite name="Disabled.Tests3" tests="1" failures="0" skipped="1" disabled="1" errors="0" time="0.296">
    <testcase status="notrun" name="DISABLED_Test_NotRun_Disabled2" time="0" classname="Disabled.Tests3">
       <disabled message="TestDisabled3" />
    </testcase>
  </testsuite>
</testsuites>

ContinuaCI_UnitTest_DisabledTests
ContinuaCI_UnitTest_DisabledTests.png919×278 10.3 KB

Regards
Julian

#2

Hi Julian,

Thank you for reporting these issues.

As you have observed, we are not taking empty columns into account in the unit tests report. We also are ignoring the disabled attribute when parsing JUnit results.

We are now working on fixing both of these issues for the next version.

1 Like
#3

Hi Julian,

The latest version - just released includes updates to fix these issues.

1 Like
#4

Hi Sparky,

We’ve installed the new build and it looks very nice. Especially with the disabled/skipped tests. Thank you!

1 Like