VSoft Technologies Blogs

rss

VSoft Technologies Blogs - posts about our products and software development.


Continua CI 1.8 Released



We are pleased to announce that Continua CI 1.8 has been released. It was actually released a couple of days ago, but for anyone who missed it, here is a heads up with and overview of the new features. We'd also like to thank all those who downloaded the beta - the time and effort spent reporting issues helped us to fix some important bugs and is most appreciated.

Version 1.8 adds the following features which build upon all the improvements and fixes made to version 1.7.1.

Dashboard Filtering


We've added a new filter box to the dashboard so you can quickly find the configuration (or project) that you are looking for as you type. Use the shortcut key F on the dashboard pages to focus on the filter box and start typing.

Dashboard filtering


Shared Resources


Many of you have requested more control over the number of builds which can run concurrently for some configurations. This may be to restrict the number of times a particular tool is run due to a license, memory or processor limit, or to prevent concurrency issues with multiple build stages simultaneously writing to the same file, folder or network resource.
You can now allocate quotas to named Shared Resources and specify that builds and stages must acquire a lock on the Shared Resource before running. If all locks are allocated, then the build or stage will wait on the queue until a lock is released.

Shared resources can be associated with the server or a particular agent in the Administration pages.

Agent shared resources

Agent shared resources are acquired when selecting an agent to run a stage. Continua will select the agent with the largest available quota of each shared resource.

Stage shared resource locks

Server shared resources can also be acquired when selecting an agent, or while on the build queue after evaluating configuration conditions.

Shared resource lock configuration condition

We hope you find that shared resources can provide many different ways to control the build process.


Requeue Build


Sometimes a build may fail due to an offline network resource, or some logical error in the stage workflow. Up until now, your only option was to re-run a build for the same branch heads. If any new changesets had been committed to the branch since that build, then you are out of luck.

The new Requeue Build button on the Build View page allows you to requeue an existing build using the same changesets, variables and queue options. Any changes to the configuration such as stage actions or repositories are taken into account and used for the new build.

Requeue build button

You can also change the priority, comment and variables before requeuing the build.

Requeue build options menu item

Clicking on the “Build requeue options” menu item will open the Queue Options dialog.

Requeue options dialog


Persist Build Variables


Another common request has been to persist variable values from one build to another build. This may be to keep a count of builds on a particular branch or to flag that some actions have been completed in one build and do not need to be repeated.

Continua CI takes a copy of configuration and project variables at the start of each build. These copies are referred to as build variables. Any changes to build variables are normally discarded when the build finishes and cannot be used by other builds.

First tab of Persist Build Variable build event handler dialog

The new Persist Build Variable build event handler allows you to save the value of the build variables when specific events happen in the build timeline. This is stored as the value of the configuration variable. Subsequent builds will then pick up this revised value and use it as the initial value of the build variable.

Second tab of Persist Build Variable build event handler dialog

As Continua CI allows multiple builds to run concurrently, it is important to control when the variables are overwritten. A later build may run faster and finish before a build which started earlier, causing unexpected results.  You can optionally state that a variable should not be persisted if the configuration variable has been modified (e.g. by another build) since a specified build event, such the build start.

Options tab of Persist Build Variable build event handler dialog

You can also prevent concurrency issues by using this feature in conjunction with shared resource locks.


Other New Features


  • You can now set the Variables display order of variable prompts on the Queue Options dialog.
  • We have provided buttons for cloning Triggers, Repositories and Build Event Handlers.
  • Configuration Conditions can now be disabled.
  • All actions which run external processes now have a Timeout (in seconds) setting.
  • We have also added a new Cake build runner action and a new VSTest unit testing action.

Automise 5 Released!

Today we are delighted to release the Automise 5, which now includes a gift. Automise Run-time is now free.

As explained in the Beta announcement, our continuous delivery cycle has worked well for Automise 4. Producing constant stream of updates and fixes. Now, with significant updates to the internal workings for Automise today we are releasing Automise 5.

What's new in Automise 5

Stepping Engine

We have undertaken a major rewrite of the internal stepping engine for Automise 5. This has reduced the moving parts, while also enabling extra features to be implemented. In the end this will mean your projects will run faster, consume less resources, and also providing some extra tools for debugging projects.

Action List Dependencies

Action Lists can now list other action lists that they depend on. For example this allows specifying a UploadAndClean Action List that depends on the Clean and Upload Action Lists. When UploadAndClean is run, if the Clean and Upload Action Lists have not been run they will be.

Action List Dependencies

Step into included projects

Previously, debugging include project actions meant running the entire included project when stepping over the action. In Automise 5, debugging allows for stepping into the included project. This will open the included project, if is not already open, and start to debug that project.

Breakpoint Conditions

We have also added to the debugging experience with breakpoint conditions. These work like break point conditions in Visual Studio. They present options for stopping the executing of a script when variables have certain values, or a condition specified becomes true. Conditions can also be the number of passes over the breakpoint.

Action List Dependencies

IDE Themes (Light and Dark)

After five years we thought it was time Automise got a new coat of paint. We have implemented two new themes, a light and dark theme (defaulting to the dark on first run up). Choose your side wisely…

Action List Dependencies Action List Dependencies

Action List Out Parameters

Action Lists parameters can now be defined as an out parameter. These parameters can have their value set inside the action list. On exiting the action list the calling Run Action List action will set the variable assigned as the out variable with that value. This effectively adds the ability to have action lists work as functions and return values calculated in the action list.

Project Formats

Let’s face it, xml files are difficult to diff. To make diffs easier Automise 5 has introduce two major updates to the Automise project file structure.

   1. A new DSL project file format (the new default format) (atp5)

   2. A new XML project file format (atx5)

New Actions

We have done some major work to bring new Azure and Amazon S3 actions to you. Some of these being:

  • Amazon S3 - Added bucket delete object, bucket list objects, bucket list, upload directory, and download folder actions.
  • Azure Actions - Added Login, Logout, Config Mode actions.
  • Azure Group Actions - Added Group Create, Group Delete, Group List, Group Log Show, Group Set, Group Show actions.
  • Azure VM - Added VM Capture, VM Create, VM Deallocate, VM Delete, VM List, VM List, VM Quick Create, VM Restart, VM Start, VM Stop actions.
  • Azure Storage - Added Storage File Copy Show, Storage File Copy Start, Storage File Copy Stop, Storage File Delete, Storage File Download, Storage File List, Storage File Upload actions.

New License Manager

Automise 5 introduces a new license manager that allows you to download licenses directly from the store. It will be presented to you on first load, or when no valid license has been found. You can also get to it from the Help menu.

All that you will require to download a license from the store is your store credentials. It will then log in for you, and download a list of licenses that will work for the current version of Automise. If you had a current Automise 4 subscription as of 28 March 2016, you will already have an Automise 5 license waiting for you in your store account.

In addition there is a simpler way to get Trial license, all that you require is a valid email address to receive a verification code on.




This version adds several new features which build upon all the improvements and fixes made to version 1.7.1.

Dashboard Filtering


We've added a new filter box to the dashboard so you can quickly find the configuration (or project) that you are looking for as you type. Use the shortcut key F on the dashboard pages to focus on the filter box and start typing.

Dashboard filtering


Shared Resources


Many of you have requested more control over the number of builds which can run concurrently for some configurations. This may be to restrict the number of times a particular tool is run due to a license, memory or processor limit, or to prevent concurrency issues with multiple build stages simultaneously writing to the same file, folder or network resource.
You can now allocate quotas to named Shared Resources and specify that builds and stages must acquire a lock on the Shared Resource before running. If all locks are allocated, then the build or stage will wait on the queue until a lock is released.

Shared resources can be associated with the server or a particular agent in the Administration pages.

Agent shared resources

Agent shared resources are acquired when selecting an agent to run a stage. Continua will select the agent with the largest available quota of each shared resource.

Stage shared resource locks

Server shared resources can also be acquired when selecting an agent, or while on the build queue after evaluating configuration conditions.

Shared resource lock configuration condition

We hope you find that shared resources can provide many different ways to control the build process.


Requeue Build


Sometimes a build may fail due to an offline network resource, or some logical error in the stage workflow. Up until now, your only option was to re-run a build for the same branch heads. If any new changesets had been committed to the branch since that build, then you are out of luck.

The new Requeue Build button on the Build View page allows you to requeue an existing build using the same changesets, variables and queue options. Any changes to the configuration such as stage actions or repositories are taken into account and used for the new build.

Requeue build button

You can also change the priority, comment and variables before requeuing the build.

Requeue build options menu item

Clicking on the “Build requeue options” menu item will open the Queue Options dialog.

Requeue options dialog


Persist Build Variables


Another common request has been to persist variable values from one build to another build. This may be to keep a count of builds on a particular branch or to flag that some actions have been completed in one build and do not need to be repeated.

Continua CI takes a copy of configuration and project variables at the start of each build. These copies are referred to as build variables. Any changes to build variables are normally discarded when the build finishes and cannot be used by other builds.

First tab of Persist Build Variable build event handler dialog

The new Persist Build Variable build event handler allows you to save the value of the build variables when specific events happen in the build timeline. This is stored as the value of the configuration variable. Subsequent builds will then pick up this revised value and use it as the initial value of the build variable.

Second tab of Persist Build Variable build event handler dialog

As Continua CI allows multiple builds to run concurrently, it is important to control when the variables are overwritten. A later build may run faster and finish before a build which started earlier, causing unexpected results.  You can optionally state that a variable should not be persisted if the configuration variable has been modified (e.g. by another build) since a specified build event, such the build start.

Options tab of Persist Build Variable build event handler dialog

You can also prevent concurrency issues by using this feature in conjunction with shared resource locks.


Other New Features


  • You can now set the Variables display order of variable prompts on the Queue Options dialog.
  • We have provided buttons for cloning Triggers, Repositories and Build Event Handlers.
  • Configuration Conditions can now be disabled.
  • We have also added a new Cake build runner action

Automise 5 Beta

Today we are delighted to release the Automise 5 BETA, which contains our new Stepping Engine and Action List Dependencies as the headline features.

For five years we have updated and improved Automise 4 through our continuous delivery cycle. This has worked well. Allowing for improvements to actions (like FTP\SFTP\FTPS suite) to come out gradually and consistently. Allowing everyone to pick and choose at which point to take feature updates.

The Automise 5 signals a major "tick" to this regular flow of updates. The majority of these updates are at the core of what Automise does to solve your automation challenges.

What's new in Automise 5

Stepping Engine

We have undertaken a major rewrite of the internal stepping engine for Automise 5. This has reduced the moving parts, while also enabled extra features to be implemented. In the end this will mean your projects will run faster, consume less resources, while also providing some extra tools for debugging projects.

Action List Dependencies

Action Lists now allow for listing of other Actions Lists they are dependent on. Dependencies are always run before the action lists which depend on them. For example this allows specifying a UploadAndClean Action List that depends on the Clean and Upload Action List. When UploadAndClean is run, if the Clean and Upload Action Lists have not been run they will be.

Action List Dependencies

Step into included projects

Due to the previous version of the stepping engine stepping into included projects was not possible. Instead the user had to wait for the included project to complete before continuing with debugging. Stepping into included projects with Automise 5 will now open the included project, and continue stepping from inside the included project.

Breakpoint Conditions

Another addition to the debugging experience is breakpoint conditions. These allow stopping the executing of a script at a certain point in time. Conditions can be a number of passes over the breakpoint, or when a variable equals a certain value.

Action List Dependencies

IDE Themes (Light and Dark)

After five years we thought it was time Automise got a new coat of paint. We have implemented two new themes, a light and dark theme (defaulting to the dark on first run up).

Action List Dependencies Action List Dependencies

Action List Out Parameters

Action Lists now allow for retrieving any number of values from them. A variable assigned to the out parameter on the Action List will be given the value of that parameter when the Action List has completed. This will allow for more Action Lists that generate values for use else where in the Automise Project.

Project Formats

Since the start of Automise the project files have used XML for their structure. As Automise has grown, so too have the elements in the projects XML file. This has placed more strain on those left to diff versions of Automise projects.

To aleavate this challenge Automise 5 has introduce two major updates to the Automise project file structure.

   1. A new DSL project file format (the new default format)

   2. A new XML project file format

The new Automise DSL structure is concise, and very simple to diff.

project
begin
    projectid = {04710B72-066E-46E7-84C7-C04A0D8BFE18}
    Action List
    begin
        name = Default
        Action Listid = {E6DE94D6-5484-45E9-965A-DB69885AA5E2}
        rootaction
        begin
            action.group
            begin
                id = {D860420B-DE46-4806-959F-8A92A0C86429}
            end
        end
    end
end
The new Automise XML structure is a great deal less verbose than the older format.
<?xml version="1.0" encoding="UTF-8"?>
<Automise>
    <project>
        <projectid>{6A717C24-D00F-4983-9FD0-148B2C609634}</projectid>
        <Action List>
            <name>Default</name>
            <Action Listid>{E6DE94D6-5484-45E9-965A-DB69885AA5E2}</Action Listid>
            <rootaction>
                <action.group>
                    <id>{D860420B-DE46-4806-959F-8A92A0C86429}</id>
                </action.group>
            </rootaction>
        </Action List>
    </project>
</Automise>

 

New Actions

Not much to report here, most of the focus has been on the Stepping engine and the IDE. We do have some updates to AWS EC2 and Azure in progress, they will most likely be added in an update when they are ready. 

How do I get the Beta?

Links to the beta downloads will be published to the Automise Downloads page. 

What if I find a bug?

Email support (please added Beta to the subject). When reporting an issue, be sure to include the beta build number and details about your environment. Please test with the latest beta build before reporting bugs. 

We are particularly keen for people to load up their existing projects from older (ie 4 or earlier) versions of Automise, save them in AT5 format, and load them again and confirm that everything loaded ok. 

When will it be released?

When it's ready ;) Seriously, though, we expect the release to happen in the next few weeks. Automise 5 is based on FinalBuilder 8, which has been out for several months now and is quite stable. 

Code Signing Changes for 2016

Microsoft announced some time ago that windows 7 & higher would no longer trust anything that is code signed with an SHA1 (https://en.wikipedia.org/wiki/SHA-1 ) certificate as of 1st Jan 2016. The reason for this is well documented, SHA1 has become increasingly vulnerable and is no longer secure enough to be trusted.

What do I need to do?

First things first, check your code signing certificate. If it's current, and uses SHA1, contact your certificate issuer for a replacement SHA256 certificate. Most issuers have a formal process for this since this is something they have known about for a while and should not charge extra (KSoftware/Comodo do not charge). In our case, our certificate was renewed in Nov 2014 and was already an SHA256 certificate. 

What if I need to support Windows XP/Server 2003?

Windows XP & Server 2003 do not support SHA256, so this deprecation of SHA1 does not apply to those versions of windows. If you sign with your SHA256 certificate using the SHA256 digest algorithm, you will find you code is not trusted on those versions of windows. The trick is to use the SHA1 digest algorithm.   

So do I need separate installers for XP and Windows 7+ ?

Well that's one way to do it, but you can support XP and windows 7+ with a single installer or exe, by signing twice, with SHA1 and SHA256. 

NOTE: If you are a long time FinalBuilder user and still using the Authenticode action, then don't, it's been deprecated for some time as it uses the deprecated capicom.dll api. The only reason we haven't removed it is to avoid errors when you load your old projects. The correct actions to use for code signing are the SignTool actions.

Recent versions of Signtool.exe  include a switch (/as) to append a signature ( the default operation is to replace the primary signature). I believe the windows 8.1 sdk  was the first version to include this option (and other related options).

In my experiments, I found that you need to sign with SHA1 first, then SHA256. The reason for this is that WinXP only looks at the first signature and would not recognise the timestamps from any RFC3161 timestamp servers that I tried. The signtool options that allow adding additional signatures (/as for signing, /tp for timestamping) only work with RFC3161 compliant timestamp servers, so the SHA1 signature and timestamp must be done first since we can't use /as or /tp with a non RFC3161 timestamp server. 

Sign, then TimeStamp

Whilst signtool can sign and timestamp in a single operation (and the SignTool Sign action in FinalBuilder can too), I prefer to do the timestamp step separately. The reason for this is that signing rarely fails (typically only when the certificate has expired or you get the password wrong!), but timestamping fails often, because the timestamp server may be unreachable or it just has some issue and doesn't respond correctly.

By doing the timestamp operation separately, we can retry if timestamping fails. Often, just a few seconds between retries is enough (unless your internet connection is down), and there is always the option of using a different timestamp server.  

So the order of events is :

Sign SHA1.
Sign SHA256 (with append signature /as).
Timestamp SHA1 - using older style authenticode timestamp server.
Timestamp SHA256 (/tp with index 1 ) - using an RFC3161 compliant timestamp server.

Show me how!

I have created an examples repository on github : FinalBuilder Examples - you can find a nice example there showing how to double (optionally) sign and then timestamp (with retries).  This example requires FinalBuilder 8.0.0.1490 or later (added support for /tp option on timestamp action). 

References

https://www.comodo.com/e-commerce/SHA-2-transition.php
http://social.technet.microsoft.com/wiki/contents/articles/32288.windows-enforcement-of-authenticode-code-signing-and-timestamping.aspx\