Import JUnit Tests problems

continua-ci
in-progress

(Julian) #1

Hi there,

I stated in another topic that we try to import Google Test XML output with the JUnit import action in continua.
I got the new version (1.9.0.430) and that works great to use the file name as assembly value. But I found some other issues while looking at our test results.

First, if we don’t specify a namespace (just the classname) for every testcase, continua has problems with the test result table view. As you can see in the image below, only the one test with a specified ‘Namespace’ has the correct layout (all other tests have a Fixture name specified which is not displayed).

Another issue we’ve experienced is with disabled tests (not shelved). We tried different ways, used disabled or skipped attribute, but all tests are always displayed as ‘Success’. The UnitTest section in the build “Details” page also doesn’t show a value for “disabled” tests (only shelved), which leads me to believe that continua doesn’t support disabled tests. Is our import test behavior working as expected for continua?

Used xml:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="4" failures="1" disabled="1" errors="0" timestamp="2019-05-08T10:09:24" time="7.171" name="AllTests">
  <testsuite name="Disabled.Tests1" tests="1" failures="0" skipped="1" disabled="1" errors="0" time="0.296">
    <testcase status="notrun" name="DISABLED_Test1_NotRun_Skipped" time="0" classname="Disabled.Tests1">
      <skipped message="TestSkipped1"><![CDATA[Disabled.Tests1]]></skipped>
    </testcase>
  </testsuite>
  <testsuite name="Disabled.Tests2" tests="1" failures="0" skipped="1" disabled="1" errors="0" time="0.296">
    <testcase status="notrun" name="DISABLED_Test2_NotRun_Disabled" time="0" classname="Disabled.Tests2">
      <skipped message="TestSkipped2"><![CDATA[Disabled.Tests2]]></skipped>
    </testcase>
  </testsuite>
  <testsuite name="Disabled.Tests3" tests="1" failures="0" skipped="1" disabled="1" errors="0" time="0.296">
    <testcase status="notrun" name="DISABLED_Test_NotRun_Disabled2" time="0" classname="Disabled.Tests3">
       <disabled message="TestDisabled3" />
    </testcase>
  </testsuite>
</testsuites>

Regards
Julian


(Dave Sparks) #2

Hi Julian,

Thank you for reporting these issues.

As you have observed, we are not taking empty columns into account in the unit tests report. We also are ignoring the disabled attribute when parsing JUnit results.

We are now working on fixing both of these issues for the next version.