VSoft Technologies Blogs

rss

VSoft Technologies Blogs - posts about our products and software development.


Delphi versions 2007 or later include MSBuild support. This means that you can (in theory at least) use MSBuild to build your Delphi Projects. I say "in theory" because in reality your milage may vary. The MSbuild support has improved somewhat since D2007. There are still issues relating to IDE bugs and configuration inheritance (even in XE4) and some other issues with dproj files that have been "upgraded" through several versions of Delphi. Delving into the dproj file to fix issues is not for the faint of heart, the easiest way to resolve those issues is to delete the dproj and allow the Delphi IDE to create a new one. 

Delphi XE2 added the ability to set the version info, before that you had to resort to patching the exe after it was built, or using your own resource file for version info and building that somehow before building the delphi project. 

This is the main reason that FinalBuilder still uses the delphi command line compiler (dcc32.exe/dcc64.exe) directly. Of course, FinalBuilder works well with Continua CI too!

So, all of those issues asside, lets take a look at getting a Delphi project to build using MSBuild and Continua CI. I'm going to use XE2 in this example, and the project we will be building is an open source test framework on GitHub called DUnitX (it's a work in progress). If you are not familiar with Continua CI, then I recommend taking a look at the Tutorial on the Continua CI Wiki. I'll be skipping stuff that will be obvious if you already use Continua CI or have read through the tutorial. 

First thing first, we need our delphi project in a source code repository that Continua CI can use. DUnitX is on GitHub, so I created a Git Repository on my Continua CI Configuration. Since DUnitX depends on Delphi-Mocks for it's own unit tests, I also added a repository in Continua CI for it. 

Notice that I have the "Fetch Other Remote Refs" option check, this allows Continua CI to see GitHub Pull Requests (I will expand on this in a future post). 

Continua CI creates a new Build Workspace folder for each build (yes, each build is a clean build). The Repository Rules are how we determine where the source code will end up in our Workspace folder. The default rules will create a folder called Source, and a folder for each repository under the Source folder. That's fine for this example so we'll stick with that. 

Now that we have our source code available, let's add an MSBuild Action to our Build Stage

On the MSBuild Action Dialog, we set the Project File to : $Source.DUnitX$\Tests\DUnitXTest_XE2.dproj

Note the $Source.DUnitX$ - this means use the Default Repository folder for the DUnitX Repository, at runtime it will evaluate to <workspace folder>\Source\DUnitX. If you use custom Repository Rules then adjust the path accordingly. 

Set the Target to Build, the Configuration to your desired config (ie, Debug or Release).

For Delphi XE2 or later, Use MSBuild 3.5, for earlier versions use MSBuild 2.0.

You can override delphi config settings, the list of settings doesn't appear to be documented anywhere I could find, however if you open :

(for XE2) C:\Program Files (x86)\Embarcadero\Rad Studio\9.0\bin\CodeGear.Delphi.Targets 

in notepad you see a bunch of elements like DCC_BplOutput. These can be set on the MSBuild Action's properties tab, in our case we need to tell the compiler where to find Delphi-Mocks, so we'll add it to the search path DCC_UnitSearchPath :

Note the $$, because Continua CI use $ as the delimeter for object references, so it needs to be escaped by doubling it. 

Delphi's MSBuild support relies heavily on Environment Variables, which is why it ships with a batch file (rsvars.bat) to set them in the command prompt. So we need to set those enviroment variables when invoking MSbuild.exe, we do that on the Environment Variables tab of the MSBuild Action,

Since the action actually runs on the Agent, we need to tell MSBuild where on the Agent to find delphi. That's done by referencing Agent Properties . The properties are created when the Agent runs the Property Collectors defined on your Continua CI Server. Continua CI ships out of the box with property collectors already defined for Delphi 7 to XE4. 

These property collectors are run on each agent, and when the collector finds what it is looking for, it creates a property on the agent  :

The Continua CI Server uses these Agent properties when working out which agents are capable of running the Stage. For some actions, this is automatic, but in this case Continua CI has no way of knowing that you need Delphi XE2 on the agent (since we're just running MSBuild), so we need to help it out by adding an Agent Requirement on our Stage :

When the Continua CI Server is searching for an Agent to run our Delphi Project on, it will only select an agent that has the Delphi.XE2.Path property. 

I hope that's enough to get you started. As I said at the start of this post, the MSBuild support in Delphi is somewhat flakey, especially in earlier versions of Delphi. If you get weird things happening, try creating a new .dproj (just rename the old one, and open the dpr in the IDE) and keep your configurations simple. If after pulling your hair out for a few days you don't get anywhere, you can always use FinalBuilder!


With the rise of  Distributed Version Control systems in recent years, the use of branching has exploded. Developers using DVCS think nothing of creating new branches,  a common pattern being the use if Feature Branches and Personal branches.

In older Continuous Integration Servers (like our own FinalBuilder Server), managing builds on branches was somewhat problematic. In Continua CI, if you use Git, Mercurial or Subversion, working with branches is easy.

For branch aware Repository types in Continua CI, you can select which branches to monitor, either a Single Branch, All Branches or By Pattern (Subversion only supports Single Branch or By Pattern).

If we select By Pattern, the UI changes to allow us to specify the pattern we want to match on. The pattern is a Regular Expression that matches the branch names (as seen & presented by Continua CI). In the example below, the Continua CI repository would see commits in branches "master", and any branches that start with feature- 

This is only half the story though. Changing the branches that the Continua CI Repository will monitor only controls which commits Continua CI knows about , but it doesn't control which branches will be built. 

In most cases, you would just leave the repository monitoring all branches, because it is Repository Triggers which control which branches are built.

Repository Triggers point to a single Repository (you can however define as many repositories and triggers as you like), and they allow you monitor commits from All Branches, the Default Branch or a Pattern matched Branch

So lets say we want to control how our build is done when using a feature branch. All we need to do is define a Trigger and specify the patten for the feature branch names

Notice that we can also define the values of Configuration Variables which will be set when the trigger queues the build. What this means is we can use the same Configuration to build dev/release/feature branches, and enable and disable parts of the build based on the variables set by the trigger (using the flow control actions in the stage editor). So for example, if our release branches all started with release-

The Branch Pattern Matching feature in Continua CI makes building Feature Branches simple.


It's been a long time coming, but the release of Continua CI, our next generation Continuous Integration server product is almost here. Continua CI will replace FinalBuilder Server (FBS), in other words, FBS 7 is the last Major release of FinalBuilder Server. Of course we will continue to support FinalBuilder Server 7 for some time, but no new features will be added to it.

Licensing wise, customers with current Software Assurance (on FinalBuilder Server User licenses) will automatically receive Continua CI licenses when it is released (using 2 User licenses = 1 Concurrent Build License). I detailed how Continua CI licensing will work in this blog post.

It's fair to say that FinalBuilder Server has some major limitations, the main one being scalability. The architecture of FBS is such that moving it forward to provide better performance and scability would be next to impossible, so we decided to start from scratch, taking into account what customers were asking for, where the industry were heading, and looking for better ways to aproach continuous integration in general.

Continua CI is written using C#, .NET 4, ASP.NET MVC4 + extensive use of jQuery. It uses a database engine (PostgreSQL or SQL Server 2008+) unlike FBS which uses .net serialization (which has been problematic).

Feature FinalBuilder Server Continua CI
Build Runners FinalBuilder Only FinalBuilder, Ant, Nant, MSBuild, Visual Studio (more will be added)
Version Control Systems Accurev, Alien Brain, Bazaar, Starteam, CVS, File System, Git, Mercurial, Perforce, PlasticSCM, Subversion, SurroundSCM, Vault, SourceSafe, TFS Bazaar, FileSystem, Git*, Mercurial*, Perforce, Subversion*, SurroundSCM, TFS, Vault (Others will be added based on demand.. except sourcesafe... let it die people!)
VCS Branch Support* Single Branch *All Branches, Single Branch or *By Pattern (depends on VCS type)
VCS Checkout/Get Optional, can be handled by FB script. Clean checkout into clean workspace for each build. Source is cached on Server and agents.
Issue Tracker support None FogBugz, Bugzilla,Jira, Youtrack, custom.
Stages, Conditions Single FinalBuilder Script Unlimited Stages, unlimited build runners.
Concurrent Builds Only of different projects, builds of projects currently running are queued. Concurrency based on license and agent(s) configuration.
Scalability Single Server, requires full server install on other machines when single server is loaded too heavily. Central Server, 1 or more agents installed on as many machines as you like.
Licensing Per User licensing. Per Concurrent Build - Free install includes 1 Concurrent build, 1 local agent, unlimited users. At least 1 concurrent build license is needed to allow remote agents.
Security Model Simple Users & Roles. Users, Groups, Roles, Permissions, Access Control Lists (global, project and configuration levels)
Authentication Limited Active Directory support, Forms Authentication only. Forms only, Mixed or Windows only Authentication
Platforms Windows Only. Windows Only - other Platforms in the future.
(*) - Branch monitoring only available on Git, Mercurial and Subversion.

Migrating to Continua CI


Key Differences


Whilst both Continua and FinalBuilder Server (FBS) are both Continuous Integration Servers, they are quite different in how they operate. Continua CI's design is based on feedback from FinalBuilder Server customers, and with a view to making it appeal to more users (FBS is only of use to FinalBuilder Users). The differences in concepts between the two products make it difficult to create an comprehensive automated migration from FBS to Continua. There is a conversion tool that provides a "leg up" in the conversion, but some manual intervention will be needed (more on that in a future post).


Working with FinalBuilder Projects


Perhaps the biggest difference for FBS users is where the FinalBuilder Project Files live. In FBS, the FinalBuilder project files (.fbp7/.fbz7) are managed by FBS and each FBS project references a single FB projects. In Continua, your FinalBuilder Project files must be in a version control Repository that is referenced by your Configuration. Continua is not tied to FinalBuilder in any way, FinalBuilder is just one of a number of different Build Runner Actions available.


Where and when the Build runs


FinalBuilder Server projects all run on the same machine, the one where FBS is installed. This unfortunately limits how many builds can be run concurrently. FBS also doesn't allow multiple builds of the same project to run concurrently, so for example a CI build triggered by a checkin, and a manually started build for the same project cannot run at the same time. The main reason for this is that the working folder and source code are managed by the FinalBuilder project.

Continua CI uses Build Agents to run the builds. Agents can be installed on the same machine as Continua Server, or on other machines. Each agent can run multiple concurrent builds (limited only by hardware resources and software licenses), and each build get's its own working folder and copy of the source code (which is managed by Continua). When a build is queued in Continua, it's requirements (compilers & other tools needed) are evaluated against the available Agents capabilities, and a suitable agent is selected. The build will run when a suitable agent is available to run the build. This may be immediately or at a later time. If no suitable agent is available, then the build will stay on the queue until one becomes available or the build is cancelled by a user.


Continua CI and FinalBuilder


Continua includes a FinalBuilder Action which will run any FinalBuilder Project which is available in the build's workspace. To get the FinalBuilder project into the build workspace, it needs to be in a Repository which is associated with the Configuration. Note that this action only supports FinalBuilder 7.0.0.1795 or later, if you are using older versions of FinalBuilder, you can still use them with the Execute Program Action. FinalBuilder needs to be installed on each Agent machine (the agent will detect that it is installed, and report that to the server).


FinalBuilder Project Variables

Since Continua lacks the tight integration with FinalBuilder that FBS has, it has no knowledge of the internals of the FinalBuilder Project and that includes Project Variables. The FinalBuilder Action in Continua allows you to set the values of those project variables from the action, but you will need to know what those project variables are. Continua is unable to inspect the FinalBuilder Project to determine the varianbles needed as it has no access to the FinalBuilder Project file at design time. We are currently looking at ways to make this easier.

Continua CI and the future


Of course, what will be released soon is only the start of what we have planned for Continua CI. We are planning an aggressive release cycle, with frequent updates. We have a long list of features we plan to implement. Continua CI is built on a modular platform, and we will be releasing other modules over time.


Documentation


Documentation site
Continua CI Concepts
Migrating from FinalBuilder Server
Setting up a real world project

Take Continua CI for a spin


The beta download links are pinned at the top of the Continua CI General Discussion forum

What if I find a bug?

Please post as much information as you can about your environment, the build of Continua CI you are using and the bug in the Continua CI Bug Reports forum.


Introducing Continua CI - Beta

We have now opened up the Beta program for Continua CI. 

What is Continua CI

Continua CI is our next generation Continuous Integration server product. This product will replace FinalBuilder Server once it's released.

Continua CI is a far more advanced CI server than FinalBuilder Server, it's not tied to FinalBuilder (although it works very nicely with it), Continua CI supports FinalBuilder, MSBuild, ANT, NANT & Visual Studio as build engines (more will come in updates). The builds are performed by Agents, either on the same machine or on other machines as the Continua CI Server.

Version Control Support

In the initial release these version control systems will be supported :

Bazaar, Git, Mercurial, Perforce, Subversion, SurroundSCM, TFS & Vault

Note that we have absolutely no plans to add support for Source Safe(it's long overdue for retirement). We will add support for others in the future where there is demand.

System Requirements

Server :

Windows Server 2003 or higher (no windows XP), .NET 4.0, SQLServer 2008 or higher, or PostgreSQL 9.1.3 or higher (this is the default db an installed with the server)

Agent : Windows XP SP3 or higher, .NET 4.0 - other platform support is currently being investigated.

Browser : HTML5 capable browser - IE9, Google Chrome 21 or higher, Firefox 14 or higher.

Licensing

Out of the box, Free of charge, Continua supports :

  • Unlimited Users
  • Unlimited Projects
  • Unlimited Configurations
  • 1 Local Agent(on the same machine as the server)
  • 1 Concurrent Build. 

After that you can purchase concurrent build licenses. Once you have at least 1 license installed, you can use as many remote agents as you like, agents do not take up licenses, concurrent builds do.

We explored many different licensing schemes and we believe this scheme will work out better for most people. The most common complaint about FinalBuilder Server was the per user licensing. Managing user licenses is easy when you have a small local team, but for larger distributed teams it becomes problematic. Most build servers have lots of projects but they are typically only running a few projects concurrently. Also, by not licensing agents, it allows you to have agents with specific capabilities. Agents report their capabilities to the server and are chosen for a build based the requirments of the project and the agent's capabilities. Say for example you have a project that is rarely built and it uses a specific tool and only installed on one machine, installing an agent on that machine will not tie up a license, it will only use a license when that project is being built.

Moving from FinalBuilder Server

There is unfortunately no direct migration from FinalBuilder Server. The two products are very different, so unfortunately the only option at this time is to start from scratch. We do have someone working on migration scripts, but those scripts will not get you fully up and running in Continua CI. The main issue has to do with where the FinalBuilder project files live in FB Server (on the server) vs Continua CI (in your version control system).

I recommend reading this page to help understand the different concepts between the two products.

How much will I have to pay when it's released?

If you have a current FinalBuilder Server Users Software Assurance Subscription at the time of release, we will convert your FinalBuilder Server User licenses into Continua Concurrent Build licenses (2 users = 1 concurrent build license).

New Concurrent Build licenses will be priced at US$199.

How do I get it?

The beta download links are pinned at the top of the Continua CI General Discussion forum

Can I see some Documentation?

Yes, the product documentation is here. The documentation is still being written however there is fair bit there already.

What if I find a bug?

Please post as much information as you can about your environment, the build of Continua CI you are using and the bug in the Continua CI Bug Reports forum.


Introducing Delphi Mocks

Delphi has had Unit Testing support (in the form of DUnit) for many years, but until now there very little in the way of Automatic Mocking. By contrast the .NET and Java worlds have plenty of mocking frameworks to choose from. 

So what are Automatic Mocks anyway? Simply put, they are mock objects that you don't have to hand code. What's wrong with hand coding mock objects you might ask? Well nothing, really, but consider the following example, where we pass an instance of our hand coded mock object to the object under test. 

type IFoo = interface
function Bar(param : integer) : string;
end;
procedure TTestSomething.SimpleTest;
var
something : ISomething;
mockFoo : IFoo;
begin
something := TSomething.Create;
mockFoo := TMockFoo.Create;
something.UseFoo(mockFoo);
end;


 What does our mockFoo mock object actually do, what is it's behavior, how many times will Bar be called and what will it return? It's not obvious from looking at the above code, so you would have to go off and find the TMockFoo

implementation. Not so hard, but what happens when someone else comes along and adds a new unit test that also uses TMockFoo, what happens if they change the TMockFoo implementation, will that break your existing tests. This is where automatic
mocks can help, because you define the behavior and expectations on you mock object right there in the unit test method :

procedure TTestSomething.SimpleTest;
var
something : ISomething;
mockFoo : TMock
begin
something := TSomething.Create;
//create our auto mock
mockFoo := TMock.Create;
//define the behavior of IFoo.Bar
mockFoo.Setup.WillReturn('hello something').When.Bar(1);
//define our expectations of how  many times TSomething will call IFoo.Bar
mockFoo.Setup.Expect.Once.When.Bar(1);
//now lets test ISomething.UseFoo
something.UseFoo(mockFoo);
//Now verify that ISomething used IFoo correctly
mockFoo.Verify;
end;

The auto mock object is defined right there in our unit test, and the behavior is not going to change unless we change it in our unit test. This sort of mocking makes it simple to focus our testing effort on TSomething rather than TFoo (our real IFoo implementation) or TMockFoo.

One of the reasons auto mocking hasn't really happened for Delphi is the lack of detailed and easy to use runtime type information (Rtti) and the lack of runtime code generation (yes it's always been possible, but not through a well defined interface like Reflection.Emit in .NET). Auto mocking involves creating types on the fly, in the case of interfaces, creating a type and implementing an interface. That's not an easy thing to do. There are some examples of this in Delphi's SOAP code, but they are not easy to follow. Delphi XE2 introduces some new features in the RTTI that make creating interface proxies simple. TVirtualInterface creates an implementation of an interface at runtime and marshals the method calls to the OnInvoke method. 

Delphi-Mocks - https://github.com/VSoftTechnologies/Delphi-Mocks is an attempt to create an Auto Mocking framework for Delphi. It makes use of Generics and Fluent style interfaces. Currently only interfaces can be mocked, and it supports only Delphi XE2 at this time. I hope to add support for TObject mocks and earlier versions (D2009+).