Following on from my questions last week related to MSBuild and Delphi, I got the main project working OK.
However I have some associated DUnit tests , and I have used your very useful code in the DUnit-XML GitHub repo to output the results in the xUnitX format so that I can load the XML result file in via a subsequent action and continue/stop the build based on the results of the unit tests.
So my unit tests compile fine, and the EXE runs fine when I run it interactively from the File Explorer, but when run from Continua using an Execute Program action, it creates a valid xml output file but never returns control to the stage. (As the stage has not exited I can also grab a copy of the EXE before the cleanup and run this and it terminates normally.)
I have tried enabling/disabling the options on the action - Check Program Exit Code, this makes no difference.
Any thoughts as to what I should to next apart from changing the timeout on the action ?
I might have been celebrating too early. Despite no code changes, executing the DUnit tests seems very flaky. If all is Ok it will execute in under a minute. Otherwise it creates the output XML file and then goes into the weeds and Continua just sits there waiting for the executable to return.
Is there anyway to debug what is happening under the hood ?
My guess is the unit test app is putting up some sort of message or waiting for input. That’s why I said make sure it’s a console app.
In your configuration cleanup options, change it so the agent workspace is not cleaned up on failure, then you can go to that folder and run the exe with the same parameters/starting folder and see what happens.
Hmmm. Sometimes it runs in a few seconds other times it outputs the XML file but never returns. I can’t see any pattern and I have not made any changes to the underlying repos for 2 days.
I have changed the cleanup, it runs interactively 100% of the time OK but that is me running it as a logged in user.
I’ve just used the RUNAS command to run as the service account that Continua runs under (both server and agent on this machine) and it works fine. The console window that appears has no entries and after about 30 seconds it closes. The XML file is recreated.
I’ve done a work around which is to add a timeout on the run unit tests action so that there is enough time for the XML file to be created but it does not hold things up if the program does not return. That will keep my evaluation going while I work out how Continua can fit into our workflow.
In our experience, there are a couple of reasons for applications to hang when run under Continua CI
User interface - ie message boxes, prompt for inputs etc.
Antivirus - if you haven’t already done so, add an exclusion for your agent workspace folder (this also speeds up builds considerably
For DUnit - make sure you are not using the GUI logger. In our DUnitX unit tests, we use ifdef’s to decide which loggers to include, for example under CI we only ever include the xml logger, there’s no point include the console logger since we’ll be importing the xml results anyway.
We have a unit that we include in all of the Unit tests that automagically configures the output exe depending on the target (debug/release) and whether we want DUnitX formatted output.
Regardless the strange thing is that it is an intermittent problem. We have file access mocked in classes that read from INI files for instance. I’ll take a look at the anti virus settings though.
FWIW, here is the code for the unit we include, makes configuring unit tests much, much simpler
// Important: copy the following lines to the top of the *.DPR file
// DEFINE THE following Symbol to get NUnit XML output
TestInsight.DUnit; // use the TestInsight runner if the project has been setup to run this way
OutputFile := ChangeFileExt(Application.ExeName, '.xml');
if ConfigFile <> '' then
RegisteredTests.LoadConfiguration(ConfigFile, False, True);
if ParamCount > 0 then
OutputFile := ParamStr(1);
PrintReportToConsole := False;
Thanks for all your help. I will refactor the test project (it has ~300 tests in 10 units) by just bringing in 1 unit at a time, needle in haystack work but it must be something in our code
Will post an update when we track it down