VSoft Technologies Blogs


VSoft Technologies Blogs - posts about our products and software development.

Continua CI includes support for running and reporting on Unit Tests, in this post we will take a look at running DUnitX Unit Tests. If you not familiar with DUnitX, it's a newish Delphi Unit Test framework, I blogged about it recently

I'm not going to cover how to get up and running in Continua CI, but rather I'll focus on the Unit Testing support. If you are not familiar with Continua CI, take a look at this recent post which describes how to build Delphi projects with Continua CI.

Assuming you have you Continua CI Configuration all set up, lets take a quick look at how to integrate our unit tests into the build process. You need to build a console application, and make use of the xml logger. This is how the dpr of our typical DUnitX console test application might look  :

  runner : ITestRunner;
  results : IRunResults;
  logger : ITestLogger;
  xmlLogger : ITestLogger;
    //Create the runner
    runner := TDUnitX.CreateRunner;
    //add a console logger, pass in true to specify quiet mode
    //as we don't need detailed console output.
    logger    := TDUnitXConsoleLogger.Create(true);
    //add an nunit xml loggeer
    xmlLogger := TDUnitXXMLNUnitFileLogger.Create;

    //Run tests
    results := runner.Execute;

    {$IFDEF CI}
    //Let the CI Server know that something failed.
    if not results.AllPassed then
      System.ExitCode := 1;
    //We don;t want this happening when running under CI.
    System.Write('Done.. press  key to quit.');
    on E: Exception do
      System.Writeln(E.ClassName, ': ', E.Message);
      System.ExitCode := 2;

The key thing to remember, is that this application is going to be running unattended, so never use ReadLn or any sort of interaction/prompting for input etc. If I had a dollar for every "Finalbuilder/Continua CI hangs during my build" bug report in the last 13 years, I'd be a... well not rich, but a few hundred dollars better off! Notice I used $IFDEF CI above to set the exit code to 1 if not all tests pass. 

So the next thing we need to is actually get the console application building in Continua CI. I covered building delphi applications with Continua CI in earlier post, so I'll just highlight a few things specific items that we need. Firsly, if you don't have the DUnitX source code in your repository, and have configured a repositoroy for it in Continua CI, then you need to update the Search Path for your console application. If you are using MSBuild to build the console app, then it's done on the Properties tab of the MSBuild Action :

MSBuild Properties

I have DUnitX in a Continau CI repository named DUnitX, and I've used the default path to the repository in the workspace ( $Source.DUnitX$ translates to "/Source/DUnitX" in the build workspace). If you are using FinalBuilder, you need to pass that to a FinalBuilder variable in the FinalBuilder Action, I'll cover that in more detail in a future post. Notice I also set the CI define I used in my code. The other important setting, is the ExeOutput path, which much be somewhere inside the build's workspace, so I set it to $Workspace$\Output - Continua CI will translate $Workspace$ at build time to be the workspace folder for the build (each Continua CI build gets a clean unique workspace folder). 

Now it's time to add our DUnitX action, somewhere in the stage workflow after we have built the test application. 


Stage Workflow

Setting up the DUnitX action is quite simple :

DUnitX Action


We set the Test Executable to our test console app, which in the MSBuild action we configured to be output to $Workspace$\Output - and we specify where to put the xml file that DUnitX will generate (because we added the NUnit logger).  The other two options control whether the to fail the Action/Build if any tests fail or error. If you have more than one Unit Test action to run in your build process, then it's bests to leave these unchecked and use the Stage Gate feature (on the Stage Options dialog) to fail the build (more on this later). 

After running the build, the Unit Test results appear in a two places, firstly the Build Details page, which shows the totals for the build (for all unit tests run) :


Unit Test Totals

You can drill into the tests by clickong on the numbers, or by clicking on the Unit Tests tab : 

Unit Test Details

This page allows you to filter by status (click on the numbers), and filter by Fixture, Namespacve etc. The first time a test fails, it will show up inder the New Failures category and under Failures. Clicking on a Failed or Errored test expands the row to show the error message logged by the test framework :

Failing Test Details

Notice the Shelve button next to each test. If you have a test that always fails, and you don't want it to cause the build to fail, shelving the tests tells Continua CI to ignore those failures in the future, until you unshelve them.  

One last feature which I touched on, is the use of Stage Gates to fails the build. Continua CI collects a bunch of metrics during the build, and we can use those metrics in Stage Gate Conditions to fail the build if for example, there are any failing tests. Each Stage has it's own Gate Conditions, which are evaluated once the stage completes.. When the build fails at the Stage Gate, you will see the output of the condition expressions (note, the expressions below are the defaults on all stages) :

Failed at Stage Gate


When working on DUnitX recently, I wanted a simple IoC container for internal use, and didn't want to add any external dependencies to the project. So I wrote my own. My container implementation only does IoC, it does not (and nor will it ever) do dependency injection. If you need Dependency Injection, the Delphi Spring Framework includes is a comprehensive IoC/DI container (along with some other useful stuff). 

Simple IoC is a copy of the IoC container in DUnitX, extracted into it's own GitHub project. It's basically a single pas file, with a class TSimpleIoC. Lets take a look at TSimpleIoC :

TSimpleIoC = class
  constructor Create;
  destructor Destroy;override;

  //Default Container
  class function DefaultContainer : TSimpleIoC;

  //Registration methods. Register the interface type, and an implementation. 
  //Exe's compiled with D2010 will crash when these are used.
  //NOTES: The issue is due to the two generics included in the functions. The constaints also seem to be an issue.
  procedure RegisterType<TInterface: IInterface; TImplementation: class>(const name : string = '');overload;
  procedure RegisterType<TInterface: IInterface; TImplementation: class>(const singleton : boolean;const name : string = '');overload;

  procedure RegisterType<TInterface: IInterface>(const delegate : TActivatorDelegate<TInterface>; const name : string = '' );overload;
  procedure RegisterType<TInterface: IInterface>(const singleton : boolean;const delegate : TActivatorDelegate<TInterface>; const name : string = '');overload;

  //Register an instance as a signleton. If there is more than one instance that implements the interface
  //then use the name parameter
  procedure RegisterSingleton<TInterface :IInterface>(const instance : TInterface; const name : string = '');

  function Resolve<TInterface: IInterface>(const name: string = ''): TInterface;

  //Returns true if we have such a service.
  function HasService<T: IInterface> : boolean;

  //Empty the Container.. usefull for testing only!
  procedure Clear;

  //Raise an exception if Resolve would return nil.
  property RaiseIfNotFound : boolean read FRaiseIfNotFound write FRaiseIfNotFound;

There are two ways to use TSimpleIoC, either by just using the DefaultContainer class function (I wanted to call it Default, but that's a reserved word in Delphi!), or by creating an instance of TSimpleIoC and using the instance. 

A quick refresher on IoC

The main purpose of IoC is to decouple the interface from the implementation. Without IoC, I need to know (or rather the compiler does) that TMySmartService implements IMyService, and I need to instanciate TMySmartService to get an instance that implements IMyService. But what if I want to use multiple implementations?  I guess I could do this :

function GetMyService(const implname : string) : IMyService;
  if implename = 'dumb' then
    result := TMyDumbService.Create
    result := TMySmartService.Create;

But that's not very nice. What If I need to provide another implementation? This is where the use of an IoC Container simplify's things. The above code becomes this :

  mySvc : IMyService;
  mySvc := TSimpleIoC.DefaultContainer.Resolve<IMyService>;
  mySvc := TSimpleIoC.DefaultContainer.Resolve<IMyService>('dumb');

Of course we still need to tell the IoC container how to instanticiate something that implements IMyService.  

Registering Implementations

TSimpleIoC has a bunch of RegisterType overloads, the best one's of which only work on Delphi XE and up, the Delphi 2010 compiler falls over quite badly. I could have removed the D2010 support altogether, but we're using it here with DUnitX and FinalBuilder 7 (which is written in D2010). So, lets register our IMyService implementation with the container :

  TMyServiceTest = class
    procedure FixtureSetup;
procedure TMyServiceTest.FixtureSetup;

In the above example, if we called Resolve<IMyService>, a new instance of TMySmartService would be created each time. Registering multiple implementations is as simple as giving the implementations names. You can of course also register Singletons, even already instantiated implemetations as a singleton. 

Poor Man's DI

Ok, so I said Simple IoC would never do DI, but you can fudge it using an activator delegate. An activator delegate is an anonymous function that will create an instance of your implementation. You can use this as a means to pass in constructor dependencies. 

     function : IMyService
        result := TMyService.Create(TDependency.Create);

I did say fudge!

Notes for DUnitX Users

DUnitX has this IoC container, but it's called TDUnitXIoC. Note that DUnitX makes use of the DefaultContainer internally, so you should not call the Clear method on it. You would be better off creating your own container instance. 

Simple IoC is open source, get it from GitHub

But, we have DUnit....

Unit Testing in Delphi is not new, the DUnit framework has been around for many years. So why create a new unit test framework? My motivation for creating a new framework was partially frustration with the confusion over DUnit, DUnit2 and who was maintaining DUnit? Sourceforge and subversion makes it very difficult to make contributions to projects. I thought about forking DUnit and putting it up on GitHub or Bitbucket,  but my feeling was that would make it even more confusing. Add to that, DUnit was created a long time ago, it's full of IfDefs for CLR, LINUX etc, so it's probably best left alone (I don't have Kylix or Delphi.NET installed). 

DUnitX supports Delphi 2010 or later. If you need support for older versions of Delphi, I'm afraid you will have to stick with DUnit, DUnitX makes use of Generics, Attributes and Anonymous methods. D2010 support has some limitations due to compiler bugs. 

DUnitX - How does it differ from DUnit?

DUnitX is modeled after NUnit, with some ideas borrowed from xUnit as well. The terminology is different to DUnit, so it's worth looking at the differences below :

Feature  DUnit DUnitX
Base Test Class TTestCase None
Test Method Published Published or decorated with [Test]
Fixture Setup Method NA Decorated with [SetupFixture] or constructor*
Test Setup Method Override Setup from base class Decorated with [Setup]
Test TearDown Method Override Teardown from base class Decorated with [TearDown]
Namespaces Through registration parameter (string) Unit Names (periods delimit namespaces).
Data driven Tests NA Decorated with [TestCase(parameters)]
Asserts Check(X) Assert class
Asserts on Containers(IEnumerable<T>) Manual Assert.Contains*, Assert.DoesNotContain*, Assert.IsEmpty*
Asserts using Regular Expressions NA Assert.IsMatch (XE2 or later).
Stack Trace support Jcl Jcl, madExcept 3, madExcept 4, Eurekalog 7 **
Memory Leak Checking FastMM4 FastMM4 (under construction) ** 
IoC Container Use Spring or other Simple IoC container Built in.
Console Logging Built in Built in (quiet or verbose modes).
XML Logging Built in (own format) Built in - Outputs NUnit compatible xml.

* Not available in D2010 due to compiler bugs.
** Extensible,  simple api.

DUnitX is Attribute driven. Any class can be a Test Fixture. The benefit this brings is to isolate internal architecture changes to the test framework from the tests that use the framework.  Logging, stack trace support and memory leak tracking support are extensible and replacable via the built in IoC container and simple interfaces.  The XML Logger for DUnitX outputs NUnit compatible XML, which means the xml can be imported by FinalBuilder and Continua CI (which can display the results nicely).

So what does a DUnitX Test look like?

  [TestFixture('Example','General Example Tests')]
  TExampleFixture = class
    procedure IamAboutToRunSomeTests;

    procedure OKIAmDoneThanks;
    procedure SetupMyTestPlease;
    procedure TearDownMyTestPlease;
    procedure APublicTest;
    //Ignored Test will show as ignored in the log.
    [Ignore('Need to rewrite this test')]
    procedure IAmNotReadyToRunYet;
    [TestCase('Case 1','1,2')]
    [TestCase('Case 2','3,4')]
    [TestCase('Case 3','5,6')]
    procedure RunMeMoreThanOnce(param1 : integer; param2 : integer);    
    [Test(false)] //Disabled Test. Will not show in the log, will not run
    procedure DontCallMe;
    procedure IAmATest;

Can I convert my DUnit Tests to DUnitX?

DUnitX has limited support for DUnit Tests. Conversion to DUnitX is relatively simple, DUnitX.DUnitCompatibility.pas has a TTestCase class with all the Check(X) methods on it (marked as deprecated, which delegate to the new Assert methods. In the uses clause, replace 




and change the registration to 


Enabling Stack Trace Support

DUnitX has a DUnitX.StackTrace.inc file, uncomment the define for the provider you want to use. If you want to provide your own stack trace provider, then implement IStacktraceProvider and register it with 


Enabling Memory Leak Tracking

This will be similar to the Stack Trace support when fully implemented. 

Where do I get it? Can I contribute?

DUnitX is hosted on GitHub, and we welcome contributions (fork it, make the change and submit a pull request). The project is actively being worked on, with several contributors already. My hope is that someone will help out with MacOS support (we have tried to limit reliance on windows wheree possible).  DUnitX is also the test framework being use for the Delphi Unit Test Project (DUT) started by Nick Hodges. 

Delphi versions 2007 or later include MSBuild support. This means that you can (in theory at least) use MSBuild to build your Delphi Projects. I say "in theory" because in reality your milage may vary. The MSbuild support has improved somewhat since D2007. There are still issues relating to IDE bugs and configuration inheritance (even in XE4) and some other issues with dproj files that have been "upgraded" through several versions of Delphi. Delving into the dproj file to fix issues is not for the faint of heart, the easiest way to resolve those issues is to delete the dproj and allow the Delphi IDE to create a new one. 

Delphi XE2 added the ability to set the version info, before that you had to resort to patching the exe after it was built, or using your own resource file for version info and building that somehow before building the delphi project. 

This is the main reason that FinalBuilder still uses the delphi command line compiler (dcc32.exe/dcc64.exe) directly. Of course, FinalBuilder works well with Continua CI too!

So, all of those issues asside, lets take a look at getting a Delphi project to build using MSBuild and Continua CI. I'm going to use XE2 in this example, and the project we will be building is an open source test framework on GitHub called DUnitX (it's a work in progress). If you are not familiar with Continua CI, then I recommend taking a look at the Tutorial on the Continua CI Wiki. I'll be skipping stuff that will be obvious if you already use Continua CI or have read through the tutorial. 

First thing first, we need our delphi project in a source code repository that Continua CI can use. DUnitX is on GitHub, so I created a Git Repository on my Continua CI Configuration. Since DUnitX depends on Delphi-Mocks for it's own unit tests, I also added a repository in Continua CI for it. 

Notice that I have the "Fetch Other Remote Refs" option check, this allows Continua CI to see GitHub Pull Requests (I will expand on this in a future post). 

Continua CI creates a new Build Workspace folder for each build (yes, each build is a clean build). The Repository Rules are how we determine where the source code will end up in our Workspace folder. The default rules will create a folder called Source, and a folder for each repository under the Source folder. That's fine for this example so we'll stick with that. 

Now that we have our source code available, let's add an MSBuild Action to our Build Stage

On the MSBuild Action Dialog, we set the Project File to : $Source.DUnitX$\Tests\DUnitXTest_XE2.dproj

Note the $Source.DUnitX$ - this means use the Default Repository folder for the DUnitX Repository, at runtime it will evaluate to <workspace folder>\Source\DUnitX. If you use custom Repository Rules then adjust the path accordingly. 

Set the Target to Build, the Configuration to your desired config (ie, Debug or Release).

For Delphi XE2 or later, Use MSBuild 3.5, for earlier versions use MSBuild 2.0.

You can override delphi config settings, the list of settings doesn't appear to be documented anywhere I could find, however if you open :

(for XE2) C:\Program Files (x86)\Embarcadero\Rad Studio\9.0\bin\CodeGear.Delphi.Targets 

in notepad you see a bunch of elements like DCC_BplOutput. These can be set on the MSBuild Action's properties tab, in our case we need to tell the compiler where to find Delphi-Mocks, so we'll add it to the search path DCC_UnitSearchPath :

Note the $$, because Continua CI use $ as the delimeter for object references, so it needs to be escaped by doubling it. 

Delphi's MSBuild support relies heavily on Environment Variables, which is why it ships with a batch file (rsvars.bat) to set them in the command prompt. So we need to set those enviroment variables when invoking MSbuild.exe, we do that on the Environment Variables tab of the MSBuild Action,

Since the action actually runs on the Agent, we need to tell MSBuild where on the Agent to find delphi. That's done by referencing Agent Properties . The properties are created when the Agent runs the Property Collectors defined on your Continua CI Server. Continua CI ships out of the box with property collectors already defined for Delphi 7 to XE4. 

These property collectors are run on each agent, and when the collector finds what it is looking for, it creates a property on the agent  :

The Continua CI Server uses these Agent properties when working out which agents are capable of running the Stage. For some actions, this is automatic, but in this case Continua CI has no way of knowing that you need Delphi XE2 on the agent (since we're just running MSBuild), so we need to help it out by adding an Agent Requirement on our Stage :

When the Continua CI Server is searching for an Agent to run our Delphi Project on, it will only select an agent that has the Delphi.XE2.Path property. 

I hope that's enough to get you started. As I said at the start of this post, the MSBuild support in Delphi is somewhat flakey, especially in earlier versions of Delphi. If you get weird things happening, try creating a new .dproj (just rename the old one, and open the dpr in the IDE) and keep your configurations simple. If after pulling your hair out for a few days you don't get anywhere, you can always use FinalBuilder!

With the rise of  Distributed Version Control systems in recent years, the use of branching has exploded. Developers using DVCS think nothing of creating new branches,  a common pattern being the use if Feature Branches and Personal branches.

In older Continuous Integration Servers (like our own FinalBuilder Server), managing builds on branches was somewhat problematic. In Continua CI, if you use Git, Mercurial or Subversion, working with branches is easy.

For branch aware Repository types in Continua CI, you can select which branches to monitor, either a Single Branch, All Branches or By Pattern (Subversion only supports Single Branch or By Pattern).

If we select By Pattern, the UI changes to allow us to specify the pattern we want to match on. The pattern is a Regular Expression that matches the branch names (as seen & presented by Continua CI). In the example below, the Continua CI repository would see commits in branches "master", and any branches that start with feature- 

This is only half the story though. Changing the branches that the Continua CI Repository will monitor only controls which commits Continua CI knows about , but it doesn't control which branches will be built. 

In most cases, you would just leave the repository monitoring all branches, because it is Repository Triggers which control which branches are built.

Repository Triggers point to a single Repository (you can however define as many repositories and triggers as you like), and they allow you monitor commits from All Branches, the Default Branch or a Pattern matched Branch

So lets say we want to control how our build is done when using a feature branch. All we need to do is define a Trigger and specify the patten for the feature branch names

Notice that we can also define the values of Configuration Variables which will be set when the trigger queues the build. What this means is we can use the same Configuration to build dev/release/feature branches, and enable and disable parts of the build based on the variables set by the trigger (using the flow control actions in the stage editor). So for example, if our release branches all started with release-

The Branch Pattern Matching feature in Continua CI makes building Feature Branches simple.