VSoft Technologies BlogsVSoft Technologies Blogs - posts about our products and software development.https://www.finalbuilder.com/resources/blogsBlack Friday Sale 2023 - 50% off all new licenseshttps://www.finalbuilder.com/resources/blogs/postid/846/black-friday-sale-2023-50-percent-off-all-new-licensesAutomise,Continua CI,Delphi,FinalBuilderFri, 24 Nov 2023 02:27:58 GMT<p>50% OFF. No, that’s not a typo! Our first ever Black Friday sale - 50% off all new licenses - valid to midnight Tues 28th Nov (UTC).</p> <p>No coupon code required, the store will apply the discount automatically.</p> 846Code Signing with USB Tokenshttps://www.finalbuilder.com/resources/blogs/postid/845/code-signing-with-usb-tokensContinua CI,Delphi,FinalBuilderMon, 03 Oct 2022 16:40:22 GMT<p>Big changes are coming for code signing certificates in 2023. New and reissued publicly trusted organisation validation (OV) and individual validation (IV) code signing certificates will have to be issued or stored on preconfigured secure hardware by the issuing Certificate Authority (CA) and the device must meet FIPS 140 Level 2, Common Criteria EAL 4+ or equivalent security standards.</p> <p>This is already the case for EV (Extended Validation) certificates, and it presents some problems in an automated build environment. In this post we'll take a look at the issues with hardware-based certificates and how to work around them. </p> <h2>Why is this change necessary?</h2> <p>If you work in IT, you will have heard or read about the <a href="https://www.sans.org/blog/what-you-need-to-know-about-the-solarwinds-supply-chain-attack/" target="_blank">SolarWinds supply chain hack</a>. It was a big deal. It's more common than we might think - in February 2022 <a href="https://www.malwarebytes.com/blog/news/2022/03/stolen-nvidia-certificates-used-to-sign-malware-heres-what-to-do" target="_blank">NVIDIA</a> had their code signing certificates stolen and they were used to sign malware (those certificates have since expired).</p> <p>These (and other) episodes made many in the industry (Microsoft in particular) very nervous. Trust is a big deal when it comes to certificates, and that is certainly the case when it comes to certificate issuance, but there is not a lot of trust in how those certificates are secured by the developers using them. Ask anyone who has done the merry validation dance with a CA, it's not that easy to get a code signing certificate these days. With that in mind, the CA/Browser forum <a href="https://cabforum.org/2022/04/06/ballot-csc-13-update-to-subscriber-key-protection-requirements/" target="_blank">adopted a proposal</a> to change the requirements for how issued certificates are stored.</p> <p>The change makes a lot of sense - it's much harder to steal hardware than it is to steal files. </p> <h2>What does this mean</h2> <p>From 1 June 2023, all new and reissued publicly trusted OV and IV code signing certificates will have to be issued or stored on a pre-configured secure hardware device by the issuing certificate authority (CA) and the device must meet FIPS 140 Level 2, Common Criteria EAL 4+ or equivalent security standards. </p> <p>Existing OV/IV certificates will continue to work, but if you need your certificate to be reissued you may encounter issues due to key size requirements changing (so don't lose your certificate). The reality is that most certificate providers have already switched to issuing certificates on tokens (or discontinued selling OV/IV certificates). This is the end of simply downloading a pfx. </p> <h2>What are these hardware devices </h2> <p>These devices fall broadly into 3 categories:</p> <h3>Network-attached Hardware Security Modules (HSM)</h3> <p style="text-align: center"><img src="/blogimages/vincent/codesign-usb/SafeNet-Luna-Network-HSM_0.png" /></p> <p>HSM's in this class bring a lot of benefits and functionality (like key usage audit logs) - code signing is just part of that. These devices are not cheap - usually in the "if you have to ask you probably can't afford" price range! There's a reason for that steep price though. They are designed to be ultra-secure and tamper proof - open the lid and you will likely lock it up or brick it. </p> <p>CA's will charge you a premium if you BYOD (bring your own device) - expect an audit fee of around $500 - or you can employ your own suitably qualified auditor (no idea what the criteria is for that but it sounds expensive). You will also have to deal with creating a Certificate Signing Request (CSR) to send to the CA. The process varies depending on the device, and the CA websites don't offer much guidance there. </p> <h3>Cloud HSM's</h3> <p>Azure, AWS, Google and others provide cloud HSM's, a service layer in front of network-attached HSM's - you basically rent time/space on them. They typically charge a monthly fee and then a fee per cryptographic operation. CA's charge a hefty fee to create certificates for use on these services - up to $1200 - and on top of that some CA's charge per number of signings per year (no idea how they police that). You also need to do some work to configure secure access to the HSM. These services make sense if you are already running on the cloud. Like other HSM's you will need to create a CSR to send to the CA during the order/validation process</p> <p>SSL.com offer an eSigner cloud-based service (they are also a CA), but the prices will make you think twice about code signing: USD $100 per month for 10 signings, plus $10 for each additional signing. We sign every build that might escape the building and each build has several files that need signing! </p> <h3>USB tokens</h3> <p>In my research so far, the most common USB token is the Gemalto/Thales SafeNet token. The only other ones I have encountered are Yubikey (only SSL.com seems to be using those) and Certum (for which I could find very little info on the client software). The token tends to be included in the price of the certificate (I did see one case where it was not). You do not need to create a CSR (unless you are using your own existing token) as the CA loads the certificate on the device before posting it to you. The one I have (from Digicert) is a Safenet token. It's already obsolete and cannot be used for new certificates as it doesn't support the larger key size required now (newer model required).  </p> <h3>Locked In</h3> <p>One thing to note about all the possible hardware devices, whether it's yours or one you rent, is that once a certificate is installed on that device, it's private key cannot be exported. So if you decide the cloud service you are using is too expensive and want to move, well it's time for a new certificate. </p> <p>Some CA's say they cannot reissue EV's on USB tokens, whilst others provide a procedure - it's likely you will be up for a new token cost and more verification hoops to jump through. So don't lose or damage it! </p> <p>In the rest of this post, I'm only going to cover USB tokens. If you have access to network or cloud HSM's then you are probably well past this point.  </p> <h2>So, what's the problem then?</h2> <p>Different USB tokens might use different client software/drivers - but they all have one thing in common - the USB token needs to be present (i.e. plugged in to the machine) when code signing. This seemingly innocuous little USB token (which looks just like a memory stick) needs to be physically secure. If someone walks past your machine and takes it (likely thinking it's a memory stick), well you are up a creek without a paddle. My SafeNet token has a bright blue LED on the end that just screams "Take me!". Our build servers are colocated at a data centre - so leaving things like USB devices plugged in is asking for trouble. It's not like I can walk over and plug it in when needed (every day!). The data center is 300km from where I live/work.</p> <p>Add to this that build machines are typically virtual, so you are into the realm of USB passthrough. If you use Hyper-V Server (as we do), well you are bang out of luck.. not supported. I have heard that VMWare ESXI supports it just fine but have never used it. I tested with XCP-ng and did get it working, but it was a major hassle to configure (reams of commands and copying of guids).</p> <h2>USB - Remotely</h2> <p>Fortunately, there are alternatives to USB passthrough. I looked at a bunch of USB remoting products (USB over IP/UDP), and after poor results with most, I found one that works. In fact it works incredibly well, with much better pricing than the others.</p> <p>That product is <a href="https://virtualhere.com" target="_blank">VirtualHere</a> (VH). Of all the vendors I contacted, they were the only one who actually responded and answered the question I asked - "Does it support code signing tokens?". The author responded, "Actually I use my (Digicert) JC Token via VirtualHere to sign VirtualHere when I build VirtualHere inside a VM." Good enough for me to give it a try!</p> <p>The VH Server runs on Windows, Linux, MacOS, a bunch of NAS servers, even a Raspberry Pi! The server licence is locked to the host machine, so if you decide to move the USB token to another host you will need to purchase another license - but at USD$49 that probably won't break the bank!</p> <p>I installed the VH server software on my XCP-ng host - installation was trivial and took all of 2 minutes (I'm no Linux expert). I then plugged the USB token in (the server is in my mini rack at home) and installed the VirtualHere client software on my Windows machine.</p> <p>The VH client can auto detect servers, however in my case the two machines were on different subnets, so I had to manually specify it. With the trial version, a message box shows up when it first connects. The client immediately showed a tree of the USB devices plugged into the server. The SafeNet token shows up as Token JC, right-click on it and select "Auto use this device" so it connects automatically. When I did this, the familiar Windows sound indicated a device had plugged in. I already had the SafeNet software installed so it didn't prompt me for drivers etc.</p> <p style="text-align: center"><img src="/blogimages/vincent/codesign-usb/virtualhere-client.png" /></p> <p>The last step in this USB remoting journey was to install the client as a service (right click on the USB Hubs node). This can only connect to licensed servers, so leave this step until you have purchased.<br /> <br /> <strong>NOTE</strong> : I didn't make it clear before, but the VH client and token client software need to be installed on the machine where the signing takes place, ie your build machine or build agent machine. </p> <h2>Prompting for Passwords</h2> <p>Another issue with the USB tokens being present during code signing, is they also expect a human to be present - a password prompt is shown. That flies in the face of conventional wisdom - automate all the things!</p> <p>Fortunately, for the SafeNet token at least, there is a work around for this.</p> <p>Open the Safenet Authentication Client Tools, click on the Advanced View button (the gear). You may be prompted for the token password if you haven't already entered it. In the tree on the left, right-click on the certificate (under User certificates) and select Export certificate. Just to be clear here, this is the certificate without the private key (which stays on the token) - you can't use this exported certificate on a machine that does not have access to the USB token. </p> <p style="text-align: center"><img src="/blogimages/vincent/codesign-usb/safenetvalues.png" /></p> <p>In the certificate view, under Private Key, take note of the Cryptographic Provider value (likely "eToken Base Cryptographic Provider" and the Container Name (p11#xxxxxxxxxxx). You will have to manually type them out somewhere as it doesn't support the clipboard. Save those values somewhere - as you will need them in your build process for code signing.</p> <p>Whilst still in the Client tools, select Client Settings and go to the Advance tab, check the "Enable single Login" and "Enable single Logon for PKCS#11." options, and set Automatic Logoff to Never - then hit Save. You can close the client now.</p> <h2>Code Signing</h2> <p>With all that done, we can use Signtool action in FinalBuilder. The Signing option tab is where those values we saved earlier come into play.</p> <p>The only difficult one is the Private Key container. Fortunately, some clever person on StackOverflow figure out the required format:</p> <p>[{{%CSPWD%}}]=p11#xxxxxxxxxx</p> <p>I have used a variable CSPWD for the token password.</p> <p style="text-align: center"><img src="/blogimages/vincent/codesign-usb/signtooloptions.png" /></p> <p>That's it. In my tests I have run FinalBuilder from the Windows task scheduler while logged out, and from a Continua CI build agent service (again while logged out) and it worked in both instances. I did a bunch of login/out/reboot testing and it continued to work. VirtualHere has been flawless. The next step is to configure our CI agents to access the USB token over VPN. Sadly our EV token is about to expire (and we never used it once in production, our OV cert still has another year left) - so I first have to jump through the validation hoops to get a new one.</p> <p>When using this technique on a CI server, you will need to take care that only one build at a time is using the token. In Continua CI, that is trivial to achieve using a <a href="https://wiki.finalbuilder.com/display/continua/Shared+Resources" target="_blank"> shared resource lock.</a></p> <p>I would love to test out some of the cloud HSM services too, but purchasing a certificate for each one is too rich for me. If you are using any of those with a cloud-based HSM, jump on our forums and let us know your experiences. If you experiment with or get up and running using VH let us know how it went - I might do a follow up post as we all gain more experience with usb tokens and code signing.</p> <h2>Warning (added 19 Oct 2022)</h2> <p>I probably should have pointed out that most tokens are configured to lock you out after too many authentication failures. So if you are getting auth failures when setting this up, stop, and manually login to the token to reset the failed count. </p> <h2>Rant</h2> <p>CA's (and their resellers) have some of the worst websites I have ever had the displeasure of reading. Pages and pages of useless or contradictory information with links promising more information that take you around in circles. Grrrrr. </p> 845Continua CI System Server Propertieshttps://www.finalbuilder.com/resources/blogs/postid/844/continua-ci-server-propertiesContinua CI,Delphi,FinalBuilder,GeneralThu, 13 May 2021 12:59:28 GMT<p>If you are using Continua for your CI, (and if not why not?) ensure that you check out <a href="https://wiki.finalbuilder.com/x/BYDp">System Server Properties</a>. These allow access to global settings which do not fit on any existing page.</p> <p>They can be used to configure several aspects of the UI and build process to fit your team preferences. This could simply be the number items to show per page on each of the dashboard views (<em>Server.ProjectsView.*.PageSize</em>), or more complex patterns for detecting errors and warnings in actions settings (<em>Actions.Messages.*Patterns</em>). Some system server properties are rarely needed, but some can be considered essential, such as <em>Server.HostUrl</em> which can be used to ensure the links in notifications go to the correct external host name.</p> <p>We have recently added some new server properties which allow you to control the tabs on the Queue Options dialog (<em>Server.QueueOptionsDialog.*</em>) and create a banner for displaying a message to all (or a subset of) users (<em>Server.Banner.*</em>).</p> <p>Here is the result of changing <em>Server.QueueOptionsDialog.TabSequence</em> from "Variables,Repositories,Options"</p> <p><img alt="Queue Options dialog tabs - Variables first" src="/blogimages/daves/serverprops/QueueOptionsVariablesFirst.png" /></p> <p>to "Repositories,Variables,Options".</p> <p><img alt="Queue Options dialog tabs - Variables first" src="/blogimages/daves/serverprops/QueueOptionsRepositoriesFirst.png" /></p> <p> </p> <p>This is what happens to an existing banner when you change the <em>Server.Banner.MessageType</em> from "Information"</p> <p><img alt="Information Banner" src="/blogimages/daves/serverprops/InformationBanner.png" /></p> <p>to "Warning".</p> <p><img alt="Warning Banner" src="/blogimages/daves/serverprops/WarningBanner.png" /></p> <p>Server properties can be edited on the "Continua Server - Properties" page located in the "Administration" section of Continua CI. See our documentation for <a href="https://wiki.finalbuilder.com/x/BYDp">details on all the currently available server properties</a>.</p> 844Announcing the Release of Continua CI Version 1.9.2https://www.finalbuilder.com/resources/blogs/postid/840/introducing-the-release-of-continua-ci-version-192Continua CI,DelphiMon, 09 Nov 2020 06:16:32 GMT<p>We are delighted to announce that <a href="/downloads/continuaci/continua-ci-version-history-v192" target="_blank">version 1.9.2 of Continua CI</a> has passed through the beta and release candidate stages, and has now been released. Here is a reminder of the new features in v1.9.2:</p> <ul> <li><a href="#export-and-import"><b>Export and Import</b></a>: You can now export one or more configurations to a file and import them back from the file into Continua CI.</li> <li><a href="#requeuing-stages"><b>Requeuing Stages</b></a>: Requeue a failing stage without restarting the build.</li> <li><a href="#multiple-daily-cleanup-rules"><b>Multiple Daily Cleanup Rules</b></a>: Each type of build by-product can now have a different shelf life.</li> </ul> <h2><a name="export-and-import">Export and Import</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Users with Configuration Edit permissions can now export one or more project configurations to a YAML or JSON file. This may be for backup, versioning or migration to another server.</p> <p>The export wizard has a number of steps allowing selection of one or more configurations, and also any related repositories, variables and shared resources.</p> <p><img alt="Export Wizard - Configuration Selection" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Export_Resized.png" /></p> <p>The configuration details can be exported to YAML or JSON file formats, according to your preferences for readability and differencing.</p> <p><img alt="Export Wizard - File Details" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Export-File-Details_Resized.png" /></p> <p>The resultant file is downloaded to your computer, allowing you to file it away until you need it.</p> <p><img alt="Export Wizard - Downloaded File" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Export-Download.png" /></p> <p><img alt="Export Wizard - YAML" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Export-YAML.png" /></p> <p>The import wizard also consists of several steps, allowing users with Project Edit permissions to upload a file, ...</p> <p><img alt="Import Wizard - File Selection" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Import-File-Details_Resized.png" /></p> <p>choose which items in the file to import and whether to overwrite any existing matching items of create new items.</p> <p><img alt="Import Wizard - Configuration Selection" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Import-Configurations_Resized.png" /></p> <p>The import runs in a transaction, so if any modified file content fails validation it will rollback...</p> <p><img alt="Import Wizard - Import Failed" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Import-Failed_Resized.png" /></p> <p>allowing you to make changes and retry.</p> <p><img alt="Export Wizard - Import Complete" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Import-Complete_Resized.png" /></p> <h2><a name="requeuing-stages">Requeuing Stages</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Sometimes a build stage may fail due to external influences. It could be that a file server was offline, network connectivity was down, or a file was locked for access. If it has taken several long stages to get to this point, then having to run the whole build again from the start can be a pain.</p> <p>The last stage of a completed build can now be requeued, providing that it has failed, stopped or errored, and the server workspace is intact.</p> <p>If no parts of the server workspace have been removed by the cleanup process, then a Requeue Stage button will be shown after the last stage in the Stages list on the Build page.</p> <p><img alt="Action list categories" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Requeue-stage-button.png" /></p> <p>This allows you to requeue and execute the stage again!</p> <p><img alt="Action list search" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Requeue-stage-running.png" /></p> <p>You can also optionally make changes to the stage actions and requeue the stage with the latest changes.</p> <p><img alt="Stages" class="dropShadow" src="/blogimages/daves/v1.9.2-release/Requeue-stage-dialog.png" /></p> <h2><a name="multiple-daily-cleanup-rules">Multiple Daily Cleanup Rules</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Every build that is executed within Continua CI stores information in the server's workspace, such as artifacts and build logs, and entries in the database. These by-products are vital for executing your build process and tracking build information, however, they can also take up considerable disk space over time and have a negative impact on database performance. The cleanup settings define the shelf life for the build by-products.</p> <p>Up until now, the cleanup settings have been quite limited - you could set up a single policy per configuration defining the build age and build limits for cleaning up either the database, the workspace, or both. Often, however you would want to cleanup the workspace files to save space, well before removing the build from the database. This update allows you to define multiple cleanup rules, with different shelf lives for each type of build by-product.</p> <p><img alt="Cleanup rules" class="dropShadow" src="/blogimages/daves/v1.9.2-release/CleanupRules_Resized.png" /></p> <p>Each rule can include one or more by-product to clean up.</p> <p><img alt="Cleanup rules dialog" class="dropShadow" src="/blogimages/daves/v1.9.2-release/CleanupRule.png" /></p> <p>Download the installers for Continua CI v1.9.2 from the <a href="/downloads/continuaci" target="_blank">Downloads</a> page</p> <style type="text/css">ul.horizontal { overflow: auto; margin: 10px 0 0 0; } ul.horizontal li { float: left; margin-left: 2em } .syntaxhighlighter { background-color: #eee !important; margin-top: 0 !important; padding: 10px; outline-style: dashed; outline-width: 1px; outline-color: #666; width: 97% !important; } .syntaxhighlighter .line.alt2 { background-color: #eee !important; } .syntaxhighlighter table td.code { overflow-y: hidden !important; } .syntaxhighlighter .plain, .syntaxhighlighter .plain a { color: #639099 !important; } h1 { margin-bottom: 0.6em !important; } .up-link { float: right } img.dropShadow { -webkit-filter: drop-shadow(5px 5px 5px #222); filter: drop-shadow(5px 5px 5px #222); margin-bottom: 1em; } </style> 840Introducing Continua CI Version 1.9.2 Betahttps://www.finalbuilder.com/resources/blogs/postid/839/introducing-continua-ci-version-192-betaContinua CI,DelphiSun, 30 Aug 2020 14:12:26 GMT<p>We are delighted to announce a new <a href="/downloads/continuaci/continua-ci-version-history-v192" target="_blank">beta release</a> of Continua CI. We have added the following new features:</p> <ul> <li><a href="#export-and-import"><b>Export and Import</b></a>: You can now export one or more configurations to a file and import them back from the file into Continua CI.</li> <li><a href="#requeuing-stages"><b>Requeuing Stages</b></a>: Requeue a failing stage without restarting the build.</li> <li><a href="#multiple-daily-cleanup-rules"><b>Multiple Daily Cleanup Rules</b></a>: Each type of build by-product can now have a different shelf life.</li> </ul> <h2><a name="export-and-import">Export and Import</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Administrators can now export one or more project configurations to a YAML or JSON file. This may be for backup, versioning or migration to another server.</p> <p>The export wizard has a number of steps allowing selection of one or more configurations, and also any related repositories, variables and shared resources.</p> <p><img class="dropShadow" alt="Export Wizard - Configuration Selection" src="/blogimages/daves/v1.9.2/Export_Resized.png" /></p> <p>The configuration details can be exported to YAML or JSON file formats, according to your preferences for readability and differencing.</p> <p><img class="dropShadow" alt="Export Wizard - File Details" src="/blogimages/daves/v1.9.2/Export-File-Details_Resized.png" /></p> <p>The resultant file is downloaded to your computer, allowing you to file it away until you need it.</p> <p><img class="dropShadow" alt="Export Wizard - Downloaded File" src="/blogimages/daves/v1.9.2/Export-Download.png" /></p> <p><img class="dropShadow" alt="Export Wizard - YAML" src="/blogimages/daves/v1.9.2/Export-YAML.png" /></p> <p>The import wizard also consists of several steps, allowing you to upload a file, ... </p> <p><img class="dropShadow" alt="Import Wizard - File Selection" src="/blogimages/daves/v1.9.2/Import-File-Details_Resized.png" /></p> <p>choose which items in the file to import and whether to overwrite any existing matching items of create new items.</p> <p><img class="dropShadow" alt="Import Wizard - Configuration Selection" src="/blogimages/daves/v1.9.2/Import-Configurations_Resized.png" /></p> <p>The import runs in a transaction, so if any modified file content fails validation it will rollback...</p> <p><img class="dropShadow" alt="Import Wizard - Import Failed" src="/blogimages/daves/v1.9.2/Import-Failed_Resized.png" /></p> <p>allowing you to make changes and retry.</p> <p><img class="dropShadow" alt="Export Wizard - Import Complete" src="/blogimages/daves/v1.9.2/Import-Complete_Resized.png" /></p> <h2><a name="requeuing-stages">Requeuing Stages</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Sometimes a build stage may fail due to external influences. It could be that a file server was offline, network connectivity was down, or a file was locked for access. If it has taken several long stages to get to this point, then having to run the whole build again from the start can be a pain.</p> <p>The last stage of a completed build can now be requeued, providing that it has failed, stopped or errored, and the server workspace is intact.</p> <p>If no parts of the server workspace have been removed by the cleanup process, then a Requeue Stage button will be shown after the last stage in the Stages list on the Build page.</p> <p><img class="dropShadow" alt="Action list categories" src="/blogimages/daves/v1.9.2/Requeue-stage-button.png" /></p> <p>This allows you to requeue and execute the stage again!</p> <p><img class="dropShadow" alt="Action list search" src="/blogimages/daves/v1.9.2/Requeue-stage-running.png" /></p> <p>You can also optionally make changes to the stage actions and requeue the stage with the latest changes.</p> <p><img class="dropShadow" alt="Stages" src="/blogimages/daves/v1.9.2/Requeue-stage-dialog.png" /></p> <h2><a name="multiple-daily-cleanup-rules">Multiple Daily Cleanup Rules</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Every build that is executed within Continua CI stores information in the server's workspace, such as artifacts and build logs, and entries in the database. These by-products are vital for executing your build process and tracking build information, however, they can also take up considerable disk space over time and have a negative impact on database performance. The cleanup settings define the shelf life for the build by-products.</p> <p>Up until now, the cleanup settings have been quite limited - you could set up a single policy per configuration defining the build age and build limits for cleaning up either the database, the workspace, or both. Often, however you would want to cleanup the workspace files to save space, well before removing the build from the database. This update allows you to define multiple cleanup rules, with different shelf lives for each type of build by-product. </p> <p><img class="dropShadow" alt="Cleanup rules" src="/blogimages/daves/v1.9.2/CleanupRules_Resized.png" /></p> <p>Each rule can include one or more by-product to clean up.</p> <p><img class="dropShadow" alt="Cleanup rules dialog" src="/blogimages/daves/v1.9.2/CleanupRule.png" /></p> <p>Download the installers for Continua CI v1.9.2 Beta from the <a href="/downloads/continuaci" target="_blank">Downloads</a> page</p> <style type="text/css">ul.horizontal { overflow: auto; margin: 10px 0 0 0; } ul.horizontal li { float: left; margin-left: 2em } .syntaxhighlighter { background-color: #eee !important; margin-top: 0 !important; padding: 10px; outline-style: dashed; outline-width: 1px; outline-color: #666; width: 97% !important; } .syntaxhighlighter .line.alt2 { background-color: #eee !important; } .syntaxhighlighter table td.code { overflow-y: hidden !important; } .syntaxhighlighter .plain, .syntaxhighlighter .plain a { color: #639099 !important; } h1 { margin-bottom: 0.6em !important; } .up-link { float: right } img.dropShadow { -webkit-filter: drop-shadow(5px 5px 5px #222); filter: drop-shadow(5px 5px 5px #222); margin-bottom: 1em; } </style> 839Daily Builds with Continua CIhttps://www.finalbuilder.com/resources/blogs/postid/838/daily-builds-with-continua-ciContinua CI,Delphi,Deployment,TriggersSun, 24 May 2020 09:22:55 GMT<p>Generally, at VSoft, we like to build. So we build every commit and this allows us to look back at our build history and see which changes caused the build to fail. We use manual stage promotion to prevent every build being released until we decide that it is ready to go.</p> <p><img alt="Build promotion" src="/blogimages/daves/dailytriggers/BuildPromotion.png" /></p> <p>Many teams like to trigger a build at the end of each day, or during the night, compiling the work for the day in one single package.</p> <p>The obvious choice for this scenario is the Daily Trigger. This can be set to run a build at a specific time every day, or just weekdays - even just weekends for those with alternative lifestyles.</p> <p><img alt="Build promotion" src="/blogimages/daves/dailytriggers/DailyTrigger.png" /></p> <p>But what if the team is just having a design day, is off on a team building excursion or, perish the thought, a day of meetings! No commits are made, but the daily build still runs even though there are no changes. One possible solution is to use a Discard condition.</p> <p><img alt="Discard condition" src="/blogimages/daves/dailytriggers/DiscardCondition.png" /></p> <p>This will prevent the build running if there are no changes since the last build.</p> <p><img alt="Build being discarded" src="/blogimages/daves/dailytriggers/DiscardCondition.gif" /></p> <p>Another option has been added to Continua CI recently. The Quiet Period setting on Repository Triggers now allows you to enter an End Time rather than an Interval.</p> <p><img alt="Quiet period end time on repository trigger" src="/blogimages/daves/dailytriggers/EndTimeRepositoryTrigger.png" /></p> <p>Any builds triggered from a repository change are then queued right through the day until the specified end time. Any additional changes added to the configuration repositories during the day are added to the queued build, and when the end time comes up, the build executes on the latest changeset.</p> <p><img alt="Build waiting on quiet period end time" src="/blogimages/daves/dailytriggers/QuietPeriodTrigger.png" /></p> <p>If you're going home earlier than the end time and want stuff deployed already, you can swiftly end the quiet period at the click of a button. Using a repository trigger in this way means that you can ignore changes to some files, changesets with a specific comment, or commits from certain users.</p> <p><img alt="Trigger with user exclusion" src="/blogimages/daves/dailytriggers/TriggerUserExclusion.png" /></p> <p> - like that hands-on manager who thinks of his commit count as a key performance indicator.</p> <p><a href="https://www.commitstrip.com/en/2016/05/09/when-the-pm-fixes-a-bug/" target="_blank"><img alt="'When the PM fixes a bug' cartoon by commitstrip.com" src="/blogimages/daves/dailytriggers/Strip-Quand-les-PM-se-mettent-au-code-650-finalenglish-1.jpg" /></a> </p> 838Introducing Continua CI Version 1.9.1 Betahttps://www.finalbuilder.com/resources/blogs/postid/835/introducing-continua-ci-version-191-beta.NET,Continua CI,Delphi,DeploymentWed, 01 May 2019 07:00:08 GMT<p>This new <a href="/downloads/continuaci/continua-ci-version-history-v191" target="_blank">beta release</a> includes substantial improvements to the expressions engine including new several expressions objects and functions. We have also made some updates to the stage editor, implemented automatic report generation for some reporting actions, and added several new deployment actions providing support for Docker, Azure, SQL packages, File Transfer and SSH.</p> <p>Continue reading for details of all the new features.</p> <ul> <li><a href="#expressions-engine">Enhanced expressions engine</a></li> <li><a href="#stage-editor-changes">Stage editor changes</a></li> <li><a href="#deployment-actions">New premium deployment actions</a></li> <li><a href="#other-actions">Other new and updated actions</a></li> <li><a href="#automatic-reporting">Automatic reporting</a></li> </ul> <h2><a name="expressions-engine">Enhanced expressions engine</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>The expression engine in Continua CI evaluates expression objects and variables denoted with $ and % characters. It also provides auto-completion suggestions when typing such expressions into expression fields. This has now been overhauled to include function return types, chaining of functions, nesting functions as function parameters, selection and filtering of collections and many improvements to expression parsing. We have added several new functions, objects and collections to give access to more values and allow you to manipulate those values.</p> <p><img alt="Expression in Set Variable action list categories" src="/blogimages/daves/v1.9.1/SetVariableAction.png" /></p> <p>You can now, for example, use the following expression to get the time that the penultimate build stage finished;</p> <pre class="brush:javascript;toolbar:false;gutter:false;"> $Build.Stages.Item($Build.Stages.Count.Decrement()$).Finished.ToLongTimeString()$</pre> <p>combine the result of multiple flags by chaining functions, as in this expression;</p> <pre class="brush:plain;toolbar:false;gutter:false;"> $Build.HasErroredStages.Or($Build.HasFailedStages$).Or($Build.HasWarnings$)$</pre> <p>or use the following expression to get the comment of the first build changeset in the build containing the word 'merge' (ignoring case):</p> <pre class="brush:plain;toolbar:false;gutter:false;"> $Source.SuperFancyRepo.Changesets.First(Comment, Contains, "merge", true).Comment$</pre> <p>We have also included functions to get the value of a variable as a type, allowing you to use properties or functions on the variable value.</p> <p>You can, for example, now use the following expression to get the abbreviated day of the week from a variable entered using a DateTime prompt;</p> <pre class="brush:javascript;toolbar:false;gutter:false;"> $Utils.GetDateTime(%DateTimeTest%).DayOfWeek.Substring(0, 3)$</pre> <p>use expressions to do some more complex maths on a Numeric variable;</p> <pre class="brush:plain;toolbar:false;gutter:false;"> $Utils.GetNumber(%NumberTest%).Floor().Modulus(10).Multiply(100)$</pre> <p>or get the first selected value in a checkbox select variable with this expression:</p> <pre class="brush:plain;toolbar:false;gutter:false;"> $Utils.GetString(%CheckboxSelectTest%).SplitWithQuotes(",").First()$</pre> <p>You can see a full list of available expression objects, collection and functions on the <a href="https://wiki.finalbuilder.com/x/C4DmAQ">Expression Objects page</a> of the documentation.</p> <p>Auto-completion has also been revamped so show more information in the suggestions list. A list of parameters with types is now shown for each for each function. Descriptions are also displayed on mouse over for each object, collection and function in the suggestions list. We have also removed some annoying quirks with expression auto-completion where the cursor would end up in the wrong place or end characters would be added in the wrong place.</p> <h2><a name="stage-editor-changes">Stage editor changes</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>As Continua CI matures, the number of actions (and categories) has increased. This can make it more difficult to find the action you need. We have therefore redesigned the action list.</p> <p>The list of categories has been pulled up into a drop down menu with all actions listed below by default.</p> <p><img alt="Action list categories" src="/blogimages/daves/v1.9.1/ActionListCategories.png" /></p> <p>The filtering of actions using the search box is now fuzzier, using partial and keyword matches.</p> <p><img alt="Action list search" src="/blogimages/daves/v1.9.1/ActionListSearch.png" /></p> <p>Stage buttons now resize (up to a maximum) to fit the stage name. If you stage names are short, this means you can fit more stages into your browser width. If your stage names are long, then the text will no longer escape the stage borders. Really long stage names which do not fit the maximum stage button size will now be truncated.</p> <p><img alt="Stages" src="/blogimages/daves/v1.9.1/Stages.png" /></p> <p>All actions now include a Validate button to allow you to check that all fields have valid values before saving.</p> <h2><a name="deployment-actions">New premium deployment actions</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>We have added a set of premium actions which can be used for deploying the results of your build. The following actions can only be used if you have purchased one or more concurrent build licenses.</p> <p><b>File Transfer action:</b> This allows you to upload files to a remote server via FTP, FTPS and SFTP. <a href="https://wiki.finalbuilder.com/x/GADc" target="_blank" title="Further information on File Transfer action"><img src="/images/info.png" /></a></p> <p><b>SSH Run Script action:</b> This can be used to run a script or list of commands on an SSH server. <a href="https://wiki.finalbuilder.com/x/KIAJAQ" target="_blank" title="Further information on SSH Run Script action"><img src="/images/info.png" /></a></p> <p><b>Azure actions:</b> Several new actions are available to allow you to deploy web apps, function apps, files and blobs to Azure. <a href="https://wiki.finalbuilder.com/x/OwBJAg" target="_blank" title="Further information on Azure actions"><img src="/images/info.png" /></a></p> <p><img alt="Azure actions" src="/blogimages/daves/v1.9.1/AzureActions.png" /></p> <ul class="horizontal"> <li>Create Azure Resource Group</li> <li>Delete Azure Resource Group</li> </ul> <ul class="horizontal"> <li>Create Azure App Service Plan</li> <li>Delete Azure App Service Plan</li> </ul> <ul class="horizontal"> <li>Create Azure Web App</li> <li>Deploy Azure Web App</li> <li>Upload Azure Web App</li> <li>Control Azure Web App</li> <li>Delete Azure Web App</li> </ul> <ul class="horizontal"> <li>Create Azure Function</li> <li>Deploy Azure Function</li> <li>Delete Azure Function</li> </ul> <ul class="horizontal"> <li>Create Azure Storage Account</li> <li>Get Azure Storage Account Keys</li> <li>Delete Azure Storage Account</li> </ul> <ul class="horizontal"> <li>Create Azure Storage Container</li> <li>Delete Azure Storage Container</li> </ul> <ul class="horizontal"> <li>Upload Azure Blob</li> <li>Delete Azure Blob</li> </ul> <ul class="horizontal"> <li>Create Azure File Share</li> <li>Delete Azure File Share</li> </ul> <ul class="horizontal"> <li>Create Azure Directory</li> <li>Delete Azure Directory</li> </ul> <ul class="horizontal"> <li>Upload Azure File</li> <li>Delete Azure File</li> </ul> <p><b></b></p> <p><b>Docker actions:</b> These new actions are available to allow you to build, deploy and manage Docker containers. <a href="https://wiki.finalbuilder.com/x/AwAPAg" target="_blank" title="Further information on Docker actions"><img src="/images/info.png" /></a></p> <ul class="horizontal"> <li>Docker Build</li> <li>Docker Command</li> <li>Docker Commit</li> <li>Docker Inspect</li> <li>Docker Pull</li> <li>Docker Push</li> <li>Docker Run</li> <li>Docker Stop</li> <li>Docker Tag</li> </ul> <p><b></b></p> <p><b>SQL Package actions:</b> These new actions allow you to create, update and export SQL Server database schemas and table data. <a href="https://wiki.finalbuilder.com/x/PwBJAg" target="_blank" title="Further information on SQL Package actions"><img src="/images/info.png" /></a></p> <ul class="horizontal"> <li>SQL Package Export</li> <li>SQL Package Extract</li> <li>SQL Package Import</li> <li>SQL Package Publish</li> <li>SQL Package Script</li> </ul> <h2><a name="other-actions">Other new and updated actions</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p><b>Extent Reports:</b> Wrapper for the Extent Reports CLI for reporting on NUnit results. <a href="https://wiki.finalbuilder.com/x/3oE6Ag" target="_blank" title="Further information on Extent Reports action"><img src="/images/info.png" /></a></p> <p><b>ReportGenerator:</b> Updated to include all the latest command line options. <a href="https://wiki.finalbuilder.com/x/WQMZ" target="_blank" title="Further information on ReportGenerator action"><img src="/images/info.png" /></a></p> <p><b>Rename Directory:</b> Does what it says on the tin.. <a href="https://wiki.finalbuilder.com/x/BgAxAg" target="_blank" title="Further information on Rename Directory action"><img src="/images/info.png" /></a></p> <h2><a name="automatic-reporting">Automatic reporting</a> <a class="up-link" href="#" title="Go to top"><img src="/images/up-icn.png" /></a></h2> <p>Currently, there are a few steps to configure when setting up a report. You have to ensure that the report files are included in the Workspace Rules and that the report is defined in the Reports section of the configuration wizard. Furthermore, it's also recommended to include the report files in the artifact rules so that you can control when they are cleaned up.</p> <p>To simplify this process, we have added a new option to automatically register the report with the server to actions which generate reports (FinalBuilder, ReportGenerator and the new Extent Reports action). Ticking this option shows a new tab where you can enter the name, description and run order of the report. When a stage completes, any report files generated by actions where this option is turned on, will automatically be copied to the server workspace. The main report file will be registered as a report and all report files will be registered as artifacts.</p> <p><img alt="FinalBuilder automatic report option" src="/blogimages/daves/v1.9.1/FinalBuilderAutomaticReport.png" /></p> <p>Download the installers for Continua CI v1.9.1 Beta from the <a href="/downloads/continuaci" target="_blank">Downloads</a> page</p> <style type="text/css">ul.horizontal { overflow: auto; margin: 10px 0 0 0; } ul.horizontal li { float: left; margin-left: 2em } .syntaxhighlighter { background-color: #eee !important; margin-top: 0 !important; padding: 10px; outline-style: dashed; outline-width: 1px; outline-color: #666; width: 97% !important; } .syntaxhighlighter .line.alt2 { background-color: #eee !important; } .syntaxhighlighter table td.code { overflow-y: hidden !important; } .syntaxhighlighter .plain, .syntaxhighlighter .plain a { color: #639099 !important; } h1 { margin-bottom: 0.6em !important; } .up-link { float: right } </style> 835Introducing Continua CI Version 1.9https://www.finalbuilder.com/resources/blogs/postid/782/introducing-continua-ci-version-19.NET,Continua CI,Delphi,General,Web Development,WindowsTue, 14 Aug 2018 13:47:08 GMT<p><img alt="" src="/blogimages/dave/ContinuaCIWizardImageSmall.png" style="border-width: 0px; border-style: solid; margin-right: 5px; margin-left: 5px; width: 55px; height: 55px;" /></p> <p>Version 1.9 is now out of beta and available as a stable release. Thank you to those of you who have already tried out the beta - especially those who reported issues.</p> <p>This version brings major changes to the notifications system. We redesigned it using a common architecture, that makes it much easier to add new notification publisher types. Where previously, only email, XMPP and private message notifications were available, there are now publishers for Slack, Teams, Hipchat and Stride. And we can now add more (let us know what you need).</p> <p><img alt="" src="/blogimages/dave/PublisherTypes.png" style="width: 626px; height: 399px; display: block; margin-left: auto; margin-right: auto; " /></p> <p>We are no longer limited to one publisher of each type. You may, for example, have different email servers for different teams on your company. You can set up two email publishers, one for each server, and set up subscriptions so that notifications from different projects go to different email servers. Likewise for different Slack workspaces, Teams channel connectors and so on.</p> <p>We have also improved the XMPP publisher to support sending notifications to rooms. Subscriptions have been improved, allowing you to specify a room and/or channel for this and other publishers.</p> <p><img alt="" src="/blogimages/dave/Subscription.png" style="display: block; margin-left: auto; margin-right: auto; width: 625px; height: 733px;" /></p> <p>User preferences have been updated allowing each user to specify a recipient id, username or channel per publisher.</p> <p><img alt="" src="/blogimages/dave/UserPreferences.png" style="width: 726px; height: 751px; display: block; margin-left: auto; margin-right: auto; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19);" /></p> <p>You can see some metrics on the throughput of each publisher (number of messages on queue, messages sent per second, average send time, etc.) on the Publishers page in the Administration area. This also shows real-time counts of any errors occurring while sending messages and also any messages waiting on a retry queue due to rate limiting or service outages. This allows you to know when you need to upgrade rate limits or make other service changes.</p> <p><img alt="" src="/blogimages/dave/PublisherMetrics.png" style="width: 990px; height: 306px; display: block; margin-left: auto; margin-right: auto; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19);" /></p> <p>The Templates page has been updated. Templates are now divided into a tab per publisher. The list of available variables for each event type has been moved to a expandable side panel.</p> <p><img alt="" src="/blogimages/dave/NotificationTemplates.png" style="width: 564px; height: 707px; display: block; margin-left: auto; margin-right: auto; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19);" /></p> <p>This release is built on .Net Framework version 4.7.2, which has allowed us to upgrade a number of third party libraries, including the database ORM and PostgreSQL drivers. This has noticeably improved performance, as well as providing us with a richer platform to build future features on. The setup wizard will prompt for you to install .Net Framework version 4.7.2, before continuing with the installation.</p> <p><img alt="" src="/blogimages/dave/InstallerFrameworkRequirement.PNG" style="width: 503px; height: 391px; display: block; margin-left: auto; margin-right: auto;" /></p> <p>Note that applications running on .Net 4.7.2 do not run on versions of Windows prior to Windows Server 2008R2 and Windows 7 SP1. We are also dropping the 32-bit server installer. This is mainly to reduce testing overheads. We will still be releasing 32-bit agents for those who are using 16-bit compilers.</p> <p>We will continue to provide bug fixes to Continua CI version 1.8.1 for while to give you time to migrate from older platforms.</p> <p> </p> <p> </p> 782Continua CI Roadmap 2018https://www.finalbuilder.com/resources/blogs/postid/776/continua-ci-roadmap-2018.NET,Continua CI,Delphi,Web DevelopmentMon, 18 Jun 2018 14:49:38 GMT<p>I'm not usually one for publishing roadmaps, mostly because I don't like to promise something and not deliver. That said, we've had a few people ask recently what is happening with Continua CI. </p> <div style="background:#eeeeee;border:1px solid #cccccc;padding:5px 10px;"><strong><span style="color:#e74c3c;">Disclaimer - nothing I write here is set in stone, our plans may change.</span></strong></div> <p><br /> A few weeks ago, I wrote up a "roadmap" for Continua CI on the whiteboard in our office. Continua CI 1.8.x has been out for some time, but we have been working on 2.x for quite a while. The length of time it is taking to get some features out is a cause of frustration in the office, that lead to a lengthy team discussion, the result was a "new plan". </p> <p>One of the reasons we had been holding features back for 2.x, is they required a change in the system requirements. Many of the third party libraries we use have dropped .net 4.0 support, so we were stuck on old versions. So rather than wait for 2.0 we will release 1.9 on .net 4.7.2. This will allow us to release some new features while we continue working on 2.0, and to take in some bug fixes from third party libraries.</p> <p>This is "The Plan" :</p> <style type="text/css">.featureTable { border: 1px solid #0071c5; border-collapse: collapse; width:100%; margin-left: auto; margin-right: auto; font-size: 14px; } .featureTable thead { background-color : #0071c5; color : White; } .featureTable td { border: 1px solid #0071c5; padding : 5px; } </style> <table align="center" cellpadding="0" cellspacing="0" class="featureTable"> <thead> <tr> <td style="width: 70px;">Version</td> <td style="width: 130px;">.NET Framework</td> <td style="width: 70px">x86/x64</td> <td style="width: 170px;">Min OS Version</td> <td style="width: 70px">UI</td> <td>Features</td> </tr> </thead> <tbody> <tr> <td>1.8.x</td> <td>4.0</td> <td>both</td> <td>Windows Server 2003R2</td> <td>MVC 4</td> <td> </td> </tr> <tr> <td>1.9.0</td> <td>4.7.2</td> <td>x64</td> <td>Windows Server 2008R2</td> <td>MVC 5</td> <td>New Notifications types</td> </tr> <tr> <td>1.9.1</td> <td>4.7.2</td> <td>x64</td> <td>Windows Server 2008R2</td> <td>MVC 5</td> <td>Deployment Actions</td> </tr> <tr> <td>1.9.2</td> <td>4.7.2</td> <td>x64</td> <td>Windows Server 2008R2</td> <td>MVC 5</td> <td>Import/Export</td> </tr> <tr> <td>2.0.0</td> <td>netcore 2.1</td> <td>x64</td> <td>Windows Server 2012</td> <td>MVC 6</td> <td>New Architecture</td> </tr> <tr> <td>3.0.0</td> <td>netcore x.x</td> <td>x64</td> <td>Windows Server 2012</td> <td>TBA</td> <td>New User Interface</td> </tr> </tbody> </table> <p> </p> <p>Let's break down this plan.</p> <h2>1.9.0 Release</h2> <p>The 1.9 Release will built on .net 4.7.2, which allowed us to take updates to a number of third party libraries, most notably NHibernate and Npgsql (postgress driver). These two libraries factor heavily in the performance improvements we see in 1.9.0. </p> <p>The major new feature in 1.9.0 will be a completely redesigned notifications architecture. In 1.8, notifications are quite limited, offering only email, xmpp and private messages. There was very little shared infrastructure between the notification types, so adding new notification types was not simple. You could only use 1 mail server and 1 xmpp server.</p> <p>In 1.9.0, notifications are implemented as plugins*, using a common architecture that made it much easier add new notification types. You can also define multiple notification publishers of the same type, so different projects can use different email servers for example.</p> <p>Notification Types :  Private message, Email, XMPP, Slack, Hipchat, Stride. More will follow in subsequent updates (let us know what you need).</p> <p>*We probably won't publish this api for others to use just yet, as it will be changing for 2.0 due to differences between the .net framework and .net core.</p> <p>If you are running Continua CI on a 32-bit machine, then start planning your migration. Supporting both x86/x64 is no longer feasable, and dropping x86 support simplifies a lot of things for us (like updating bundled tools etc).  We will continue supporting 1.8.x for a while, but only with bug or security fixes. The minimum OS version will be the same as for the .Net Framework 4.7.2 - since Windows Server 2003R2 is out of support these days, it makes sense for us to drop support for it. </p> <h2>1.9.1 Release</h2> <p>Deployment focused actions.  <br /> <br />   - AWS S3 Get/Put<br />   - Azure Blob Upload, Publish, Cloud Rest Service, Web Deploy<br />   - Docker Build Image, Push Image, Run Command, Run Image<br />   - File Transfer (FTP, FTPS, SFTP)<br />   - SSH Action<br />   - SQL Package Export, Package Extract, Package Import, Package Publish<br />   - SSH Run Script<br />   - Web Deploy</p> <p>These actions are all mostly completed, but are waiting on some other (UI) changes to make them easier to use. We'll provide more detail about these when they closer release.</p> <p><strong>Note</strong> : These actions will only be available to licensed customers, not in the free Solo Edition.</p> <h2>1.9.2 Release</h2> <p>One of the most requested features in Continua CI, is the ability to Export and Import Continua CI Projects and Configurations. This might be for moving from a proof of concept server to a production server, or simply to be able to make small changes, and import configurations into other projects. The file format will be YAML.</p> <h2>Continua CI 2.0 Release - .net core.</h2> <p>We originally planned to target .net framework 4.7 with Continua CI 2.0, but with .net core improving significantly with netcore 2.0 and 2.1, the time is right to port to .net core. The most obvious reason to target .net core is cross platform. This is something we have wanted to do for some time, and even explored with mono, but were never able to get things working in a satisfactory manner. It's our hope that .net core will deliver on it's cross platform promise, but for now it's a significant amount of work just to target .net core. So that said, our plans for Continua CI 2.0 is to get it up and running on .net core on <strong>Windows only</strong>, without losing any functionality or features. During the port we are taking note of what our Windows dependencies are for future reference. </p> <p>The current (1.8.x) architecture looks like this :</p> <p><strong><span style="font-family:Courier New,Courier,monospace;">Browser <----> IIS(Asp.net with MVC)<--(WCF)-->Service <--(WCF)-->Agent(s)</span></strong></p> <p>With .net core, it's possible to host asp.net in a service process, and that is what we have chosen to do. This cuts out the WCF layer between IIS and the service. .net core doesn't have WCF server support, and to be honest I'm not all that cut up about it ;) That said, we still need a replacement for WCF for communication between the agents and the server. We're currently evaluating few options for this.</p> <p>Continua CI 2.0  architecture currently looks like this :</p> <p><strong><span style="font-family:Courier New,Courier,monospace;">Browser <----> Service(hosting asp.net core 2.1/mvc)<--(TBD)--> Agent(s)</span></strong></p> <p>The current state of the port is that most of the code has been ported, the communication between the agents and the server is still being worked on, and none of the UI has been ported. We do have asp.net core and mvc running in the service. There are significant differences between asp.net/mvc and asp.net core/mvc, so we're still working through this, I expect it will take a month or so to go through and resolve the issues, then we can move on to new features. </p> <h3>Continua CI 2.0 - new features.</h3> <p>Rest API. This is something we had been working on for a while, but on the .net framework using self hosted Nancy (in the service, running on a separate port from IIS). Once we made the decision to port to .net core, we chose to just use asp.net rather than Nancy. Fortunately we were able to use much of what was already done with nancy on asp.net core (models, services etc) and we're currently working on this right now..</p> <p>Other features - TBA</p> <h2>Continua CI 3.0 - A new UI</h2> <p>Asp.net MVC has served us well over the years, but it relies on a bunch of jQuery code to make the UI usable, and I'll be honest, no one here likes working with jQuery! Even though we ported much of the javascript to typescript, it's still hard to create complex UI's using jQuery. The Stage Editor is a good example of this, even with some reasonably well structured javascript, it's still very hard to work on without breaking it. The UI is currently based on Bootstrap 3.0, with a ton of customisations. Of course Bootstrap 4.0 completely breaks things so we're stuck on 3.0 for now.<br /> <br /> So it's time to change tack and use an SPA framework. We've done proof of concepts with Angular and React, and will likely look at Vue before making a decision - right now I'm leaning towards React. Creating a new user interface is a large chunk of work, so work will start on this soon (it's dependent on the rest api). We're likely to look at improving usability and consistency in the UI, and perhaps a styling refresh. </p> <p>Linux & MacOS Agents - with .net core running on these platforms, this is now a possibility. We looked at this several times before with Mono, but the api coverage or behavor left a lot to be desired. We do still have some windows specific stuff to rework in our agent code, and Actions will need to filtered by platform but this is all quite doable.</p> <h2>Summing up</h2> <p>We're making a big effort here to get features out more frequently, but you will notice I haven't put any timeframe on releases outlined above, they will be released when ready. We expect a 1.9.0 Beta to be out in the next week or so (currently testing the installer, upgrades etc), and we'll blog when that happens (with more details about the new notifications features). Note that it's highly likely there will be other releases in between the ones outlined above, with bug fixes and other minor new features as per usual. We have a backlog of feature requests to work from, many of which are high priorities, so we're never short of things to do (and we welcome feature requests). </p> 776Introducing archive rules in Continua CIhttps://www.finalbuilder.com/resources/blogs/postid/766/introducing-archive-rules-in-continua-ciContinua CI,DelphiWed, 16 May 2018 14:15:12 GMT<p>In version 1.8.1.870 of Continua CI, we have added new archiving functionality to the workspace and repository rules.</p> <p>Builds can generate a lot of output files: binary library files or report files, for example. Copying a large number of these files back to the server at the end of the build can take time. Manually downloading each individual artefact from the server can be a tedious task, so compressing these files into a handy bundle makes sense.</p> <p>Previously, you would have needed to use actions, such as the Seven Zip action, in your build stages to zip these files. The compression can now be performed as part of the agent-to-server workspace rules.</p> <p>To compress a set of files in the agent workspace to an archive in the server workspace, specify a file with a zip extension on the left-hand side of a agent-to-server workspace rule.</p> <p>e.g.</p> <pre class="brush:plain; toolbar:false;gutter:false;"> Libraries.zip < Output/**.dll </pre> <p>Note that the all the usual operators are taken into account when compressing files so, in the above example, the directory structure is preserved. Likewise, using the <- operator will cause all matching files to be flattened into the root folder of the zip file.</p> <p>Doubling up with the << operator will delete any existing zip file before compressing to a new file. Without the << operator, multiple sets of files can be added to the same archive file.</p> <p>e.g.</p> <pre class="brush:plain; toolbar:false;gutter:false;"> Reports.zip < Output/**.html Reports.zip < Output/**.css </pre> <p>You can also compress files into subfolders within the zip file using the new : operator</p> <p>e.g.</p> <pre class="brush:plain; toolbar:false;gutter:false;"> Reports.zip:/css < Output/**.css </pre> <p>Once files have been compressed at the end of one stage, you may need to access the contents of zip files in the next stage. Additionally, you may wish to unpack a zip file from your repository at the start of a stage. The : operator facilitates the extracting of zip files in server-to-agent workspace rules and repository rules.</p> <p>To extract a set of files from an archive in the server workspace to a folder in the agent workspace, specify a file with a zip extension on the left-hand side of a server-to-agent workspace rule. Ensure that you follow the ‘zip’ with a : operator, otherwise the zip file will just be copied.</p> <p>e.g.</p> <pre class="brush:plain; toolbar:false;gutter:false;"> Libraries.zip: > Libraries </pre> <p>This also works for repository rules.</p> <p>e.g.</p> <pre class="brush:plain; toolbar:false;gutter:false;"> $Source.MyRepo$/Documents.zip: > Docs/Main </pre> <p>Note that the all the usual operators >, >>, -> and --> have the same meaning when extracting files as they have when copying file; signifying whether to preserve the directory structure within the zip file and whether to empty the destination folder.</p> <p>You can also specify a pattern after the : operator, allowing you to filter the extracted files.</p> <p> e.g.</p> <pre class="brush:plain; toolbar:false;gutter:false;"> Libraries.zip:/plugins/**.dll > Libraries/Plugins $Source.MyRepo$/Documents.zip:**.md > Docs/Markdown </pre> <p>See the <a href="http://wiki.finalbuilder.com/display/continua/Workspace+Rules" target="_blank">Workspace Rules documentation</a> for further details on the new archive rules syntax.</p> 766Continua CI and TLS 1.2https://www.finalbuilder.com/resources/blogs/postid/761/continua-ci-and-tls-12.NET,Continua CI,Delphi,WindowsFri, 23 Feb 2018 09:24:09 GMT<p>SSL standards are changing, and older SSL/TSL protocols are slowly being deprecated, or even turned off by some services. This post shows how to enable TLS 1.2 support in Continua CI.</p> <p>Yesterday, we started getting reports that the Github Status event handler, and the Github Status action in Continua CI had stopped working.</p> <p>Sure enough, in our testing here we were able to confirm the case. While testing this under the debugger, the error we were seeing was rather strange : "The request was aborted: Could not create SSL/TLS secure channel.". </p> <p>After some research, we found this error was due to being unable to negotiate a common protocol between the client and the server.</p> <p>Now Continua CI 1.x is built with .NET 4.0 (v2 will be on 4.7.1) - we know that .NET 4.0 doesn't support TLS 1.2, and a quick check of the github api server using <a href="https://www.ssllabs.com/ssltest/" target="_blank">SSLLabs</a> shows that they now only support TLS 1.2.</p> <p style="text-align: center;"> <img alt="" src="/blogimages/vincent/continua-tls/github-ssllabs.png" width="987" height="279" /> </p> <br /> <p> I wondered if this was announced by github - turns out they did announce this 3 weeks ago :</p> <p> <a href="https://github.com/blog/2498-weak-cryptographic-standards-removal-notice" target="_blank">Weak cryptographic standards removal notice</a> </p> <p>and yesterday they permanently disabled TLS 1.0 and 1.1</p> <p> <a href="https://github.com/blog/2507-weak-cryptographic-standards-removed" target="_blank">Weak cryptographic standards removed</a> </p> <p>Anyway, back to Continua CI. The good news is that there is a way to enable TLS 1.2 support in Continua CI. Note that this only works when running on Windows Server 2008 or later (Server 2003 does not support TLS 1.2 at all, and we will be dropping support for it with v2).</p> <p>1) Install .Net Framework 4.5 or later - since all 4.x frameworks effectively replace 4.0 - 4.5 has support for TLS 1.2</p> <p>2) Edit %ProgramFiles%\VSoft Technologies\ContinuaCI\Server\Continua.Server.Service.exe.config on the server and %ProgramFiles%\VSoft Technologies\ContinuaCI Agent\Continua.Agent.Service.exe.config on each agent - add the following line in <strong>appSettings</strong> section :</p> <pre class="brush:xml; toolbar:false;"> &lt;add key="Continua.Service.SecurityProtocolType" value="Tls|Tls11|Tls12" /&gt; </pre> Note that the key supports the following values: Ssl2|Ssl3|Tls|Tls11|Tls12|Default<br /> <p> Default = Ssl3|Tls<br /> Multiple protocols can be separated with |<br /> The value "Tls|Tls11|Tls12" will allow Continua CI to work with services that do not support or have not enabled TLS 1.2, and with services that only support TLS 1.2 .</p> <p>3) Open Regedit and add the following value to : HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319 : SchUseStrongCrypto type DWORD value 1</p> <p>4) Restart the Server and Agent Services.</p> <p>5) You may need to restart your server(s) for the registry change to take effect.</p> <p>One last note : This change also effects the communication between the Continua CI Server and agents, if you make the change on the server, make sure you make a compatible change on the agents.</p>761Continuous Integration with FinalBuilderhttps://www.finalbuilder.com/resources/blogs/postid/755/continuous-integration-with-finalbuilder.NET,Continua CI,Delphi,FinalBuilderWed, 20 Sep 2017 13:49:00 GMT<p>In this post, I'm going to look at how to structure a FinalBuilder project so that it will run on your dev machine, or on your Continua CI Server without modification. This allows the best of both worlds, develop and debug your build process on your development machine, and then later run it on your CI server.</p> <p>I'm going to assume you are familiar with FinalBuilder to speed this along.</p> <h3>Version Control</h3> <p>The very first thing we need to do is add our FinalBuilder project to our version control system. This will ensure that the Continua CI agent will be able to access the projects.</p> <p>We typically create a Build folder in each repository that has our FinalBuilder projects, installer scripts etc. Make sure you save your projects in uncompressed format (ie, .fbp8 for FinalBuilder 8), as that will make it possible to diff project file changes using your usual diff tool (we use Beyond Compare 4).</p> <p>So in a typical repository, you might have a folder structure that looks something like this :</p> <pre> \Build \Src \Docs \Help \Tools \Output </pre> <p>Note the Output folder is ignored by our version control system (via .gitignore or .hgignore for example). Don't forget to add ignores for the finalbuilder log file as you don't want to commit that to the repo.</p> <p>This is the file structure I'll be using as the example in this post.</p> <h3>Build Workspace</h3> <p>Continua CI runs each build in a separate, clean build workspace folder. The reason for this is to allow multiple builds of the same configuration to be run concurrently (for example, building different branches at the same time). This means our folder structure above will be rooted differently for each build. So to deal with this, we will define some FinalBuilder Project Variables.</p> <h3>FinalBuilder Variables</h3> <p> </p> <p style="text-align: center;"><img alt="FinalBuilder Project Variables" src="/blogimages/vincent/ci-with-fb/fb-project-variables.png" /></p> <!-- <table width="250px" style="text-align:left;border-collapse:collapse;"> <tr style="border-bottom: 1px solid #777"> <th width="30%">Name</th> <th width="20%">Type</th> <th width="40%">Default Value</th> <th width="10%">Macro</th> </tr> <tr> <td>REPO</td> <td>string</td> <td>..</td> <td>no</td> </tr> <tr> <td>WORKSPACE</td> <td>string</td> <td>..</td> <td>no</td> </tr> <tr> <td>BUILD</td> <td>string</td> <td>%REPO%\Build</td> <td>yes</td> </tr> <tr> <td>SOURCE</td> <td>string</td> <td>%REPO%\Src</td> <td>yes</td> </tr> <tr> <td>LIBS</td> <td>string</td> <td>%REPO%\Libs</td> <td>yes</td> </tr> <tr> <td>OUTPUT</td> <td>string</td> <td>%WORKSPACE%\Output</td> <td>yes</td> </tr> <tr> <td>DOCS</td> <td>string</td> <td>%REPO%\Docs</td> <td>yes</td> </tr> <tr> <td>HELP</td> <td>string</td> <td>%REPO%\Help</td> <td>yes</td> </tr> <tr> <td>TOOLS</td> <td>string</td> <td>%REPO%\Tools</td> <td>yes</td> </tr> </table> --> <p> </p> <p> </p> <p>You can of course modify these variables to suite your needs. The REPO variable has a default value of "..", we will use the Path Manipulation action to expand that relative path when our build starts, using the FBPROJECTDIR variable as the base path. This will give us the root folder of our repository, which we can then use as the basis for other path variables.</p> <p> </p> <p><strong>Side Note:</strong> FinalBuilder does not support using unrooted relative paths. The best practice is to fully expand the relative path before using it. Relative paths need to be relative to something, and in windows that is typically the Current Directory for the process, however this is not viable in a multi-threaded application. Rooted relative paths (e.g "%SOURCE%\.." ) will work in most cases, however some windows api functions do not support relative paths at all.</p> <p>The WORKSPACE variable will have the path to the Continua CI build workspace folder. Depending on how your repository and <a href="http://wiki.finalbuilder.com/display/continua/Repository+Rules">repository rules</a> are structured, this might be different.</p> <h3>Repository Rules</h3> <p>Continua CI configurations can have multiple <a href="http://wiki.finalbuilder.com/display/continua/Repositories">Repositories</a> assigned to them, for example when building FinalBuilder, we have mulitple repos on the configuration, one for our source and three others for libraries. On our dev machines, we use symlinks to map the libs folder into our main repo's path, so we end up with :</p> <pre> MainRepo\Src MainRepo\Libs <- symlink to LibsRepo\ </pre> <p>This makes it easier to deal with search paths in Delphi, as we can just use simple relative paths. With .Net, working with libraries is so much simpler due to package managers like nuget or packet.</p> <p>At the start of a Continua CI Stage, the source code is exported from the Repository Cache (a Mercurial repo) for each repository associated with the Configuration. How and what gets exported is controlled by the Repository Rules for that Stage.</p> <p>The default rules looks like this :</p> <pre> $Source$ >> Source </pre> <p>This will export all associated repos to a folder structure like this</p> <pre> workspace\source\repo1 workspace\source\repo2 </pre> <p>In this example, I'm going to change the rules to mirror the folder structure on our dev machines :</p> <p style="text-align: center;"><img alt="Repository Rules" src="/blogimages/vincent/ci-with-fb/repo-rules.png" /></p> <p>This results in the following folder structure :</p> <pre> workspace\build workspace\src workspace\libs </pre> <p>Note that you can also use rules to limit which parts of the repository are exported to the workspace, so, for example, if you have files in the repository that are not needed for the build process, you can avoid exporting them and save some time (I/O is usually the biggest overhead) using exclusion rules.</p> <p>Getting back to our FinalBuilder variables, we can see from the above folder structure, that our WORKSPACE variable will have the same value as the REPO variable. If your folder structure is different then you will need to adjust accordingly.</p> <h3>Continua CI Stage Actions</h3> <p>Ok, so now we have a basic FinalBuilder project, and Repository Rules set up to export our source to the build workspace, now it's time to call FinalBuilder. Continua CI has a FinalBuilder Action for doing just that.</p> <p style="text-align: center;"><img alt="Continua CI - FinalBuilder Action" src="/blogimages/vincent/ci-with-fb/fbaction.png" /></p> <p>We need to tell the action where our FinalBuilder project file lives. This needs to be anchored by the workspace folder, e.g $Workspace$\Build\FB\FinalBuilderBuild.fbp8</p> <p>In the Variables tab, we will check the "Automatically apply Continua CI variable values to matching FinalBuilder variables." Option. If you want to send any protected Continua variables (ie passwords, api keys etc) to FinalBuilder, then check the "Save sensitive variable values to context XML file for use by FinalBuilder." option as well. Lastly, on the Environment Variables tab, we are passing some environment variables that are used in delphi search paths to FinalBuilder.</p> <p style="text-align: center;"><img alt="Continua CI - FinalBuilder Action" src="/blogimages/vincent/ci-with-fb/fbaction-variables.png" /></p> <p>We also set some environment variables that are needed for our build process, Continua CI makes this painless.</p> <p style="text-align: center;"><img alt="Continua CI - Set Environment Variables" src="/blogimages/vincent/ci-with-fb/fbaction-env.png" /></p> <h3>FinalBuilder Project</h3> <p>Remember that we want to be able to run the project on our dev machine and from Continua CI. So some logic is required to make sure it behaves the same way in both environments.</p> <p>I added some targets to the project, Init, Build and Test.</p> <p>The Init target is a dependency of the Default target, so it will always run first. This is where we initialise the variables, detect if the project is running from Continua CI or not, and set version info. We also create the Output folder here.</p> <p style="text-align: center;"><img alt="FinalBuilder - Init Target" src="/blogimages/vincent/ci-with-fb/fb-init-target.png" /></p> <p>Back in the Default target, we call the Build and Test targets. This is wrapped in a try/finally block and we export the FinalBuilder log file to %OUTPUT%\BuildLog.html in the finally section. We can register this file as an artifact and create a report definition in Continua CI to make it easy to view in the Continua CI UI. I've covered this before in an <a href="/resources/blogs/adding-custom-reports-to-continua-ci-build-results">earlier blog post</a>, so I won't cover it here.</p> <p style="text-align: center;"><img alt="FinalBuilder - Init Target" src="/blogimages/vincent/ci-with-fb/fb-default-target.png" /></p> <p>Note that you don't have to do the whole build/test/deploy all in one FinalBuilder project. When building FinalBuilder, we do the build & test in one project, and the deploy in another that is called from the Deploy stage. This avoids the temptation to deploy from a developer workstation!</p> <p>When we build Continua CI (on Continua CI of course), we have separate Build, Obfuscate, Test, Package & Deploy Stages that use different FinalBuilder projects. This allows us to run different parts of the build process on different agent machines. The .net obfuscation tool we use is very expensive, so we only have a single license which is installed on one agent machine, so the obfuscation stage will only run on that agent machine (vm). Continua CI selects the best agent based on agent properties and the Stages Agent Requirements. I'll cover this in more detail in another blog post.</p> <p>I have added this example project to the <a href="https://github.com/VSoftTechnologies/FinalBuilder.Examples">Examples repository on GitHub</a>.</p> 755Continuous Integration Server performancehttps://www.finalbuilder.com/resources/blogs/postid/754/continuous-integration-server-performance.NET,Continua CI,Delphi,FinalBuilder,Git,Mercurial,Web DevelopmentMon, 11 Sep 2017 14:31:58 GMT<p>Continuous Integration Servers are often underspecified when it comes to hardware. In the early days of Automated Builds, the build server was quite often that old pc in the corner of the office, or an old server in the data center that no one else wanted. Developers weren't doing many builds per day, so it worked, it was probably slow but that didn't seem to matter much.</p> <p>Fast forward 20 years, and the Continuous Integration Server is now a critical service. The volume and frequency of builds has increased dramatically and a slow CI server can be a real problem in an environment where we want fast feedback on that code we just committed (even though it "worked on my machine"). Continuous Deployment only adds to the workload of the CI server.&nbsp; </p> <p>In this post, I'm going to cover off some ideas to hopefully improve the performance of your CI server. I'm not going to cover compilation, unit tests etc. (which can be where a lot of the time is spent). Instead, I'll focus on the environment, machine configuration and some settings on your Continua CI configurations.</p> <h2>Hardware Requirements</h2> <p>It's impossible to provide hard and fast specs for hardware or virtual machines, as it varies greatly depending on the expected load.</p> <p>There are a bunch of things you can tweak that may improve performance. I will touch on some key points for virtual hosts, but I'm not going to go too deep into tuning virtual hosts, that's not my area of expertise. Of course, dedicated physical machines would be the ideal, but these days, even if you do get dedicated hardware for CI/CD, it's most likely going to be as a virtual host (hyper-v or vmware) rather than an OS installed on bare metal (do companies still provision a single os on bare metal servers these days?). Virtualisation brings in a whole bunch of benefits, but it also brings with it some limitations that cannot be ignored.</p> <p>Continuous Integration environments are mostly I/O bound and Continua CI is no different in that regard. So let's look at the various resources used by CI/CD.</p> <h3>CPU</h3> <p>It's unlikely that CPU will be a limiting factor in the performance of your CI server, unless you are running other CPU intensive tasks on your server. If that's the case, then move your CI server to dedicated hardware, or at least a dedicated virtual host. </p> <p>At a minimum you should have at least 2 cores on the server. On our production server, which is a virtual machine (on Hyper-V 2012R2) with 4 virtual cores and dynamic RAM, the Windows resource monitor shows that average CPU usage usually sits around 2% when idle (no running builds, measured on the guest OS using resource monitor on the Continua Server Service). With 10 concurrent builds running, the Continua CI server service was using around 6% cpu.</p> <p>Adding another 4 cores made very little difference. The Hyper-V host machine, which is also running a bunch of agent VM's, has plenty of CPU capacity, with the average CPU usage round 5-7%. Cutting down the number of cores to 2 did make a slight difference, with the VM showing slightly higher CPU usage, however no discernible difference in build times.</p> <p> This is obviously not very scientific, but it did demonstrate (well to me at least) that CPU is not the limiting factor. I set the server VM back to 4 cores and left it at that. Our Hyper-V host machines are a few years old now, and have 7200 rpm SAS hard drives (in Raid 10) rather than SSD's (they were still too expensive when we bought the machines).</p> <p>On a Continua CI Agent, we recommend at least 2 cpu cores, and limit the concurrent builds running on the agent to 1 per core. This isn't a hard and fast rule, just a convention we adhere to here (based on some performance testing). You may want to add extra cores depending on what compilers or tools you are running during your build process. The only way to know if this is needed is to monitor cpu on the agent machine while a build is running.</p> <h3>I/O</h3> <p>The most used resources are disk read/write and network read/write. Poor I/O performance will really slow down your builds.</p> <h4>Disk</h4> <p>It goes without saying, but use the fastest disks you have available to you. If you can afford it, new generation nvme/pcie SSD's are the way to go. They are still quite expensive for larger capacities though. At the very least, use a separate disk for the operating system and software installation, and another disk for your Continua CI Server's share folder (or the agents workspace folder on agent machines). This is where most of the I/O happens during builds. This recommendation applies whether running on dedicated hardware or in a virtual machine.</p> <p> If you are running the server and agent machines on the same virtual host (as we do for our production environment) then this is very important to get right. Poor I/O performance in virtualised environments is not uncommon - having agents and the server fighting for a slice of the same I/O pie is not a good idea.<br /> <br /> On the agent machines, good disk performance is critical. When a build is started on the agent, the first thing it does is create a workspace folder. It then exports the source code from the repository cache(s) (Mercurial repo which was cloned from the server) to that folder, using the repository rules (more on this later). This workspace initialisation phase can be very slow if you have poor I/O performance.</p> <h4>Network</h4> <p>Continua CI uses networking to transfer files, repository changes etc between the server and the agents. Poor network performance will impact on build initialisation times (updating the agents repo cache, build workspace) and on build completion times (transferring workspace changes back to the server). Logging between the agent and the server will also be impacted by poor network performance.</p> <p>By default, Continua CI uses SMB to transfer files, source code (repository caches) between the server and the agents. When the server's share folder is not accessible by SMB, Continua CI will try to use SSH/SFTP (Continua CI installs it own specialised SSH service). In high latency networks (for example if the agent is remote from the server), SSH/SFTP may perform better than SMB.</p> <p>You can force an agent to use SSH/SFTP by setting the agents ServerFileTransport.ForceSSH property to true.</p> <h3>Database</h3> <p>Continua CI supports PostgreSQL (the default) or Microsoft SQL Server. If you chose to use MSSQL, we recommend running it on a separate well specified machine. MSSQL is quite heavy in it's use of RAM and disk I/O - it's best run on a machine that has been tuned to run it properly. I'm not going to go into that here, that's a whole other topic on an area that I'm definitely not an expert.</p> <p>The PostgreSQL database server that is installed by default (unless you select otherwise) with Continua CI is much more more frugal when it comes to resources. On our main Continua CI server, PostgreSQL typically using around 60MB of ram. Contrast that with SQL Server running on my dev machine, not used or touched for weeks and it's using 800MB! PostgreSQL can also be tuned, we have tried to provision it with sensible defaults that strike a balance between performance and resource usage. If you need to tune PostgreSQL, then we recommend installing your own PostgreSQL instance and pointing Continua CI at it.</p> <p>Currently the Continua CI installer doesn't provide any options for the database install location (C:\ProgramData\VSoft\ContinuaCI\PostgreSQLDB ), this is something we are looking at for a future release, that will make it possible to put the database on it's own drive. For now, it's possible to move the database to another location by using a symlink, we have a few customers who have done this successfully. Contact support if you need help with this.</p> <h2>Virtualisation Tips</h2> <h3>Virtual CPU Cores</h3> <p>In a virtual environment, it's very important not to overload your virtual host. Note that there is a difference between overloading and over allocating virtual cores. It's a common practice to allocate more virtual cores across the virtual machines than there are physical/logical cores (logical when HyperThreading is enabled), but this has to be done with the knowledge and understanding of the load on the host machine. Overloading happens when so many cores are allocated and in use that the hypervisor is unable to schedule a core to a virtual machine when needed. This results in pauses and poor performance.</p> <p>In a clustered environment this is even more important, because when a cluster node dies, or is removed for upgrades etc, virtual machines will move to another node in the cluster - if that node is already overloaded then you will soon start hearing the complaints from users!</p> <p>The best explanation I have found on how hypervisors allocate cores is this article - <a href="https://www.altaro.com/hyper-v/hyper-v-virtual-cpus-explained/">https://www.altaro.com/hyper-v/hyper-v-virtual-cpus-explained/</a> - it's Hyper-V specific (we use Hyper-V here) but much of the information also applies to VMWare.</p> <h3>Virtual Disks</h3> <p>When creating separate virtual disk volumes for your virtual machines, try to put those virtual drives on different physical drives, so they are not competing for the same I/O. Use fixed size virtual disks.</p> <h2>Continua CI Configuration Tuning</h2> <p>Continua CI is not immune to performance problems, we're always working to make it faster and consume less resources. There are however a few things that can be tuned in Continua CI to improve performance.</p> <h3>Repository Branch Settings</h3> <p>Use specific branch patterns to narrow down the number of repository files and folders which are monitored and downloaded. With repositories which use folder-based branches, such as Subversion and TFS, consider moving old branches to a separate archive folder in your repository which will not match the branch patterns. Note that you can use more than one Continua CI repository per actual repository. Some users will have multiple projects in one repository, but only need to build a single one for each configuration. Make use of relative paths, where supported by your repository type, to limit your repository to a single project folder. This can significantly speed up repository initialisation and changeset updating. </p> <h3>Repository Polling</h3> <p>Continua CI polls repositories periodically to detect new commits. Each time this occurs, Continua CI invokes the command line client for that repo, and parses the output of that process. Some clients use a surprising amount of CPU. The git client, for example, uses around 8% CPU per instance on our production server while checking for commits. Most of the time, these processes only run for a very short amount of time (when no changes are detected), however if you have a lot of repositories, these small cpu spikes can add up.</p> <p> There are a couple of options to keep this under control.</p> <p> 1) Set the appropriate polling interval for your repositories. If changes to a repository occur infrequently, then there's no point polling frequently.</p> <p>2) Set the Server property Server.RepoMonitor.MaxCheckers property. This controls how many version control client processes are spawned concurrently, the default (5) is quite conservative so you should only need to lower this on a very low spec system. If you have plenty of spare CPU capacity, then you can increase this value, however if you do then monitor CPU usage to make sure you don't overload the server.</p> <p>3) Manual polling, using post commit hooks. This reduces CPU usage on the server, by only polling for repository changes when requested and has the added benefit of reducing the load on your version control server. This does take some setting up, and depends very much on the capabilities of your version control system. I'll take a look at post commit hooks in a future blog post.</p> <h3>Repository Path Filtering</h3> <p>Repository Path Filtering is an option on all repository types, with the exception of Mercurial (*I'll explain why shortly). What this filtering does is allow you to limit which files get added to the server's repository cache. This filtering has a few benefits, less disk space used on the server and the agents, less network I/O when transferring the changes from the server to the agent, and less I/O when checking out the source into the build workspace.</p> <p>A typical use case for these rules is when you have files in your repository that rarely change and are not needed for the build process (design docs, deployment notes etc). No point adding them to the repo cache if you don't use them.</p> <p>Changes to these rules won't affect files that are already in the repository cache, but it will avoid committing changes to those filtered out files to the repo cache. The best bang for buck with these filters will come if the repository is reset (the cache is rebuilt, so filtered out files are never committed to the cache), however that can be an expensive operation, so unlike other repository settings, changing these rules will not force a reset.</p> <p>* These filters don't apply to Mercurial repositories, as we use Mercurial for our repository cache. When you point Continua CI at a Mercurial repository, it just clones it to the server (repo cache), and then clones it to the agents (repo cache) without any modifcations.</p> <h3>Repository Rules</h3> <p>Each Stage has a settings tab called Repository Rules. These rules apply when checking out the source from the agent's repository cache(s) to the build workspace. Only check out the source you need, this will improve performance. If a stage doesn't need the source at all (for example, it's only working with artifacts from previous stages), then just blank out the Repository Rules field.</p> <p>Don't leave logging of the repository rules turned on unless you are debugging the rules. Logging the files exported to the workspace can be a real performance killer.</p> <h3>Workspace Rules</h3> <p>Similar to Repository Rules, these rules control which files are transferred between the server and agent's build workspace folders, and back again. Only transfer files back to the server's workspace that you actually need, like build artifacts, reports etc. </p> <p>Don't leave logging of the workspace rules turned on unless you are debugging the rules. Logging the files transferred can be a real performance killer.</p> <h3>Actions</h3> <p>Avoid logging too much information. For example, verbose logging on MSBuild should be avoided unless debugging build issues. Output logged from actions is queued and sent back to the server to be written to the build log, this causes high network and disk I/O.</p> <h3>Disk Space</h3> <p>Disk space is quite often at a premium (especially with SSD's), and it's important to keep on top of it. This is where the Clean up Policies come into play. Continua CI allows you to specify a global clean up policy for both the server and the agents, however it can be overridden at the Project or Configuration level. The clean up policy controls how long to keep old builds and their associated workspaces around. The clean up policy is highly configurable - use it to keep control over disk space. Bear in mind that the work of cleaning up old builds is quite I/O and database intensive, so be sure to schedule it to run during a quite period</p> <h3>Anti-virus Software</h3> <p>Anti-virus software can be a major performance killer, and in instances, an application killer. If I had a dollar for every time anti-virus software turned out to be the cause of a problem with Continua CI or FinalBuilder, well that would be some serious beer money at least!</p> <p>If you have anti-virus software installed on your server or agents, be sure to add exclusions from real-time scanning for the server's share folder, and the agent's workspace folder. Add scheduled scans on those folders instead. Also, when using the bundled PostgreSQL database, add an exclusion for C:\ProgramData\VSoft\ContinuaCI\PostgreSQLDB &nbsp;- otherwise you may experience database corruption.</p> <p>You should also consider adding an exclusions for the hg.exe in the "C:\Program Files\VSoft Technologies\ContinuaCI Agent\hg" folder. We found in testing that this will speed up the processing of the repostiory rules substantially (testing with windows defender). </p> <h3>Version Control Clients</h3> <p>Avoid installing tools like TortoiseSVN or ToirtoiseHG on your server or agent machines as these programs do background indexing (for icon overlays) and can also cause file/folder access issues.</p> <h2>Wrapping Up</h2> <p>I intend to revise this post as I learn more about performance tuning, especially in a virtual environment. If you have any techniques or tweaks that helped speed up your CI Server please feel free to share them with us (and fellow users).</p>754Visual Studio 2017 15.3 Update issueshttps://www.finalbuilder.com/resources/blogs/postid/751/visual-studio-2017-153-update-issues.NET,Continua CI,FinalBuilderThu, 17 Aug 2017 10:37:26 GMT<span style="color: #ff0000;"><strong>Update 25 Sept 2017 :</strong>&nbsp;Microsoft have closed our bug report with a Wont Fix status.. seems they are too busy with other things.<br /> </span><br /> The recent Visual Studio 2017 Update (also known as VS 15.3) &nbsp;introduced a problem with command line compilation when the Lightweight Solution Load feature is enabled for the solution.&nbsp;<br /> <br /> If you are using devenv (ie you have the Use MSBuild option unchecked in FinalBuilder), and have the action set to Rebuild... be aware that while the action will succeed, nothing actually gets compiled! <br /> Just to be clear, this is <strong>not</strong> a FinalBuilder (or Continua CI) problem :<br /> <br /> <div class="reCodeBlock" style="border: 1px solid #7f9db9; overflow-y: auto;"> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\devenv.com" /rebuild "Release|Any CPU" ConsoleApps.sln</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;">&nbsp;</span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">Microsoft Visual Studio 2017 Version 15.0.26730.3.</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;"><code style="color: #000000;">Copyright (C) Microsoft Corp. All rights reserved.</code></span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">========== Rebuild All: 0 succeeded, 0 failed, 0 skipped ==========</code></span></div> </div> <br /> Turning off Lightweight Solution Load on the solution (don't forget to save the solution) results in proper compilation :<br /> <br /> <div class="reCodeBlock" style="border: 1px solid #7f9db9; overflow-y: auto;"> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\devenv.com" /rebuild "Release|Any CPU" ConsoleApps.sln</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;">&nbsp;</span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">Microsoft Visual Studio 2017 Version 15.0.26730.3.</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;"><code style="color: #000000;">Copyright (C) Microsoft Corp. All rights reserved.</code></span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">1&gt;------ Rebuild All started: Project: ConsoleApps1, Configuration: Release Any CPU ------</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;"><code style="color: #000000;">2&gt;------ Rebuild All started: Project: ConsoleApp2, Configuration: Release Any CPU ------</code></span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">3&gt;------ Rebuild All started: Project: ConsoleApp3, Configuration: Release Any CPU ------</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;"><code style="color: #000000;">4&gt;------ Rebuild All started: Project: ConsoleApp4, Configuration: Release Any CPU ------</code></span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">1&gt;&nbsp; ConsoleApps1 -&gt; C:\Users\vincent.OFFICE\Documents\Visual Studio 2017\Projects\ConsoleApps\ConsoleApps1\bin\Release\ConsoleApps1.exe</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;"><code style="color: #000000;">2&gt;&nbsp; ConsoleApp2 -&gt; C:\Users\vincent.OFFICE\Documents\Visual Studio 2017\Projects\ConsoleApps\ConsoleApp2\bin\Release\ConsoleApp2.exe</code></span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">3&gt;&nbsp; ConsoleApp3 -&gt; C:\Users\vincent.OFFICE\Documents\Visual Studio 2017\Projects\ConsoleApps\ConsoleApp3\bin\Release\ConsoleApp3.exe</code></span></div> <div style="background-color: #f8f8f8;"><span style="margin-left: 0px !important;"><code style="color: #000000;">4&gt;&nbsp; ConsoleApp4 -&gt; C:\Users\vincent.OFFICE\Documents\Visual Studio 2017\Projects\ConsoleApps\ConsoleApp4\bin\Release\ConsoleApp4.exe</code></span></div> <div style="background-color: #ffffff;"><span style="margin-left: 0px !important;"><code style="color: #000000;">========== Rebuild All: 4 succeeded, 0 failed, 0 skipped ==========</code></span></div> </div> <br /> MSBuild is not affected by this bug (<a href="https://developercommunity.visualstudio.com/content/problem/96511/visual-studo-153-devenv-rebuild-does-nothing-when.html">reported to Microsoft here</a>). Unless you have a very good reason for using devenv to compile visual studio solutions these days, just use MSBuild. <br /> <br /> &nbsp;<br />751Visual Studio 2017 RC Supporthttps://www.finalbuilder.com/resources/blogs/postid/749/visual-studio-2017-rc-support.NET,Continua CI,Delphi,FinalBuilderTue, 22 Nov 2016 14:44:36 GMTToday we released updates to <a href="https://www.finalbuilder.com/downloads/finalbuilder" target="_blank">FinalBuilder (8.0.0.2007)</a> and <a href="https://www.finalbuilder.com/downloads/continuaci" target="_blank">Continua CI (1.8.1.185)</a> which add support for Visual Studio 2017 RC.&nbsp;<br /> <br /> There were a few issues we found with Visual Studio 2017 Release Candidate. The first was, unlike every other previous version of Visual Studio, 2017 RC does not write any useful information to the registry that allows us to reliably find out where it was installed to. So at the moment, we just check the default install locations :<br /> <br /> On 64 bit windows :<br /> <br /> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE<br /> C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE<br /> C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE<br /> <br /> on 32 bit windows (do people still use it?) :<br /> <br /> C:\Program Files\Microsoft Visual Studio\2017\Community\Common7\IDE\Common7\IDE<br /> C:\Program Files\Microsoft Visual Studio\2017\Professional\Common7\IDE<br /> C:\Program Files\Microsoft Visual Studio\2017\Enterprise\Common7\IDE<br /> <br /> Yes, each edition now installs into different folders.&nbsp;<br /> <br /> As a work around for those who don't install in the default location, you can set environment variables &nbsp;<strong>MSBuild15Path</strong> (for MSBuild) and&nbsp; <strong>VisualStudio2017Path</strong>&nbsp;for Visual Studio, if we don't find VS/MSBuild in the default locations, we'll check for those environment variables. &nbsp;<br /> <br /> Another issue we found is that when building with devenv.com (ie the visual studio command line, rather than msbuild), builds take a looooong time.. it appears to hang after building, and it also seems to randomly fail with windows error&nbsp;80131623 (FatalExecutionEngineError). This has nothing to do with FinalBuilder or Continua CI, it happens on the command line as well. I have reported these issues to microsoft, hopefully they will resolve them before release. On the MSBuild Action in FinalBuilder, we were not able to get the list of targets for a dotnet core project. This is still being investigated, we do still hope to resolve this eventually.&nbsp;<br /> <br /> Note, since this work was done against a Release Candidate, things may change between now and the VS2017 RTM release (microsoft seem to have redifined what a release candidate is lately!). If you find any issues let us know.<br /> <br /> BTW, TFS 2017 support is currently being worked on for both FinalBuilder and Continua CI. We should have a VSO FinalBuilder build task available in the marketplace soon (it's close). However, getting our xaml builds working with TFS2017 is proving challenging (just like previous releases), there is no doco and it's deprecated (good riddance) and visual studio won't load our projects.. stay tuned.749Introducing Continua CI Version 1.8.1https://www.finalbuilder.com/resources/blogs/postid/748/introducing-continua-ci-version-181Continua CIThu, 08 Sep 2016 16:43:33 GMT<br /> <br /> <br /> <p>This version adds several back-end performance enhancements including significant improvements to database query caching. We have also updated various third party packages and upgraded the bundled PostgreSQl database from version 9.3.4 to version 9.5.3. What&rsquo;s more, this release builds upon all the improvements and fixes made to version 1.8.</p> <h2> New Actions</h2> <br /> <p>To keep up with the latest developments in Visual Studio, we have added several new actions for working with .Net Core projects. The DotNet Build, DotNet Pack, DotNet Publish, DotNet Restore, DotNet Run and DotNet Test actions all run the DotNet Cli command line.</p> <p><img alt="DotNet Build Action" src="https://www.finalbuilder.com/blogimages/daves/v1.8.1/DotNetBuildAction.png" /></p> <p>This release also includes a new action for importing XUnit tests from an XML report file. The sort of file you may choose to output from the DotNet Test action.</p> <p>To polish this off we have added new NPM Pack and NPM Publish actions for sharing your completed packages.</p> <h2> New Build Event Handlers</h2> <br /> <p>A powerful new addition to the Continua CI build event handlers is the HTTP Request handler which allows you to send JSON or XML to HTTP endpoints via GET, POST. PUT, DELETE, PATCH methods and extract values from the results to set as Continua CI variables.</p> <p><img alt="HTTP Request Build Event Handler" src="/blogimages/daves/v1.8.1/HTTPRequestBuildEventHandler.png" /></p> <p>This enables interaction with a various existing web services and REST APIs. You could, for example, use this feature to send a message via Slack when a build starts, tag a commit on GitLab when a build finishes or even translate a variable value to Azerbaijani using Google Translate API. If you have any in-house web services which handle HTTP requests, it is likely that these can be accessed too.</p> <p>We have also added a new GitHub Release build event handler for creating, updating and deleting GitHub releases. This also allows you to upload artifacts as GitHub release assets.</p> <h2>Other New Features</h2> <br /> <p><strong> Build Event Handler conditions</strong>. You can now control the triggering of handlers based on the value of build variables or expression objects. These values may be specified when queuing a manual build or set in the stage workflow. You could, for example, only send a GitHub Release if the number of failed unit tests $Build.Metrics.UnitTests.Failed$ is zero.</p> <p><img alt="Build Event Handler conditions" src="/blogimages/daves/v1.8.1/BuildEventHandlerConditions.png" /></p> <p><strong>Continue on failure.</strong> Builds can now be set to progress to the next stage after one stage has failed. Stage gates now include an $Stage.IsSuccessful$ condition by default. This can be removed to allow the build to continue to the next stage when a stage fails. So you can see what failed, the stages are now coloured green or red in Tile and Details dashboard views to indicate an success or failure status.</p> <p><img alt="Continue on failure" src="/blogimages/daves/v1.8.1/ContinueOnFail.png" /></p> <p><strong> New cleanup options.</strong> We have also made some changes to the server cleanup policy, giving you the options to clean up build statistics. The database category has also been split up so that build unit tests can be cleaned up without cleaning up the full build. This is important to prevent database tables growing too large when you have a large number of unit tests.</p> <p><img alt="Cleanup options" src="/blogimages/daves/v1.8.1/CleanupOptions.png" /></p>748Continua CI and FinalBuilder integration improvementshttps://www.finalbuilder.com/resources/blogs/postid/747/continua-ci-and-finalbuilder-integration-improvementsContinua CI,Delphi,FinalBuilder,FinalBuilder ServerTue, 21 Jun 2016 15:35:25 GMTContinua CI 1.8.0.176 and FinalBuilder 8.0.0.1817 (both released today) provide somewhat better integration between the two products. &nbsp;<br /> <br /> <h3>Continua CI 1.8</h3> <p> The FinalBuilder Action in Continua CI now produces an xml file that is consumed by FinalBuilder when run under Continua CI. This xml file contains all the useful information about a build, such as version numbers, changesets, variables etc.&nbsp;<br /> <br /> There is also a new option on the action that will automatically apply Continua CI variable values to matching FinalBuilder variables. What this means is, if you have a variable declared in both FinalBuilder and Continua CI with the same name, FinalBuilder's variable will automatically get the value of the Continua CI variable at runtime. This option is only available for FinalBuilder 8, if you select FinalBuilder 7 the option will not be visible. If the version of FinalBuilder 8 you are running does not support this integration, a warning will appear in the Continua CI build log.&nbsp;<br /> <br /> </p> <p style="text-align: center;"> <img alt="" src="/blogimages/vincent/continua-fb-integration/continua-fbaction.png" /> </p> <h3>FinalBuilder 8</h3> FinalBuilder 8 has two new actions. <br /> <br /> The "Is Running Under Continua" action is an If Then style action, i.e. the children of this action will only run if FinalBuilder is running under Continua CI. &nbsp;<br /> <br /> <p style="text-align: center;"> <img alt="" src="/blogimages/vincent/continua-fb-integration/isrunningcontinua.png" /> </p> <br /> <p>The other action is the Continua CI - Get Version Info action - this action will take the version info from Continua CI and apply it to a version info property set in your FinalBuilder project.;</p> <p>&nbsp;</p> <p style="text-align: center;"> <img alt="" src="/blogimages/vincent/continua-fb-integration/getversioninfo.png" /> </p> <p>The action is smart enough to use the correct version info scheme depending on whether the propertyset is a win32 or dotnet propertyset. This greatly simplifies getting the version info from Continua into FinalBuilder, no need to declare 4 variables on both sides and set them in the FinalBuilder Action in Continua CI.</p> <h3>Scripting</h3> That xml file I mentioned above is loaded into a Continua object model that is available in action script events. If you do make use of it, you should be sure to check the Continua.IsRunningUnderContinua property before using the rest of the object model (the script editor provides intellisense for the model.<br />747.NET 4.6.1, WCF and self signed X509 certificateshttps://www.finalbuilder.com/resources/blogs/postid/746/net-461-wcf-and-self-signed-x509-certificates.NET,Continua CISat, 14 May 2016 14:34:50 GMTWe have been working on moving Continua CI to .net 4.6.1 for a future release, and during this conversion (so far, mostly just updating nuget packages),&nbsp; we discovered an issue that turned out to be caused by a change to .net certificate validation. <br /> <br /> If you use self signed X509 certificates and target the .net framework 4.6.1 then you are in some fun, especially if you used makecert to generate the certificate. There is a change in behaviour in the way certificates are validated, which will leave you pulling you hair out for hours. The error you may encounter will look something like this :&nbsp;<br /> <br /> "The Identity check failed for the outgoing message. The remote endpoint did not provide a domain name system (DNS) claim and therefore did not satisfied DNS identity 'localhost'. This may be caused by lack of DNS or CN name in the remote endpoint X.509 certificate's distinguished name."<br /> <br /> If you google the error message, you will find plenty of references to using&nbsp; a DnsEndpointIdentity, only in our case, we were already doing exactly as the answers on stackoverflow were suggesting! Since we were migrating from .net 4.0 to .net 4.6.1, I started looking for info on the changes in each version of the .net framework. Eventually, I came across this page :<br /> <br /> <a class="moz-txt-link-freetext" href="https://msdn.microsoft.com/en-us/library/mt620030%28v=vs.110%29.aspx">https://msdn.microsoft.com/en-us/library/mt620030(v=vs.110).aspx</a><br /> <br /> This was the only mention of X509 certificates I could find in the change history, but it seems like it could be related, so I tried what it suggested, and low and behold, problem solved! With some further investigation of this work around, I found some issueson the wcf github repo with several references to the behaviour of certificate validation.<br /> <br /> <a class="moz-txt-link-freetext" href="https://github.com/dotnet/wcf/issues/321">https://github.com/dotnet/wcf/issues/321</a><br /> <br /> <a class="moz-txt-link-freetext" href="https://github.com/dotnet/wcf/pull/603">https://github.com/dotnet/wcf/pull/603</a><br /> <br /> So it turns out, in .NET 4.6.1 they broke the certificate validation, by only looking at the Subject Alternate Name (SAN) extension, and not falling back to the Subject Name CN field as it should. The pull request I linked to above, is for the WCF for .NET core, not 4.6.1 - so I have no way of knowing if this will be fixed for 4.6.x. &nbsp;<br /> <br /> Whilst an app.config change can work around the issue, the real fix is to generate certificates that include SAN extension.<br /> <br /> <h3>Generating a certificate without MakeCert</h3> <span>Makecert.exe, which is commonly used (on windows at least) does not support the SAN extension, M</span>akecert is deprecated, so it's unlikely there will be an update to make it able to generate certificates with the SAN certificate extension. &nbsp;Windows 8/Server 2012R2 &amp; later versions of Windows include a new powershell cmdlet to generate certificates. For Windows 7 the best option is this :<br /> <br /> <a class="moz-txt-link-freetext" href="https://gallery.technet.microsoft.com/scriptcenter/Self-signed-certificate-5920a7c6">https://gallery.technet.microsoft.com/scriptcenter/Self-signed-certificate-5920a7c6</a><br /> <br /> This appears to what the Windows 8 &amp; later cmdlet is based on, it's relatively simple to use and generates certificates that validate properly with WCF on .NET 4.6.1<br /> <br /> <pre class="brush:powershell; toolbar:true;">New-SelfSignedCertificateEx -Subject "CN=MyServer" -KeySpec Exchange -KeyUsage "DataEncipherment, KeyEncipherment, DigitalSignature" -Path c:\certs\example.pfx -Exportable -SAN "MyServer" -SignatureAlgorithm sha256 -AllowSMIME -Password (ConvertTo-SecureString "abc123dontuseme" -AsPlainText -Force) -NotAfter (get-date).AddYears(20)</pre> <br /> Note the above command is on multiple lines to make it easier to read, it can be on one line. <br />746Continua CI 1.8 Releasedhttps://www.finalbuilder.com/resources/blogs/postid/745/continua-ci-18-releasedContinua CIThu, 14 Apr 2016 13:07:29 GMT<p> <br /> <br /> </p> <p>We are pleased to announce that Continua CI 1.8 has been released. It was actually released a couple of days ago, but for anyone who missed it, here is a heads up with and overview of the new features. We'd also like to thank all those who downloaded the beta - the time and effort spent reporting issues helped us to fix some important bugs and is most appreciated.</p> <p>Version 1.8 adds the following features which build upon all the improvements and fixes made to version 1.7.1.</p> <h2>Dashboard Filtering</h2> <br /> <p>We've added a new filter box to the dashboard so you can quickly find the configuration (or project) that you are looking for as you type. Use the shortcut key F on the dashboard pages to focus on the filter box and start typing.</p> <p><img alt="Dashboard filtering" src="/blogimages/daves/v1.8/DashboardFiltering.png" /></p> <br /> <h2>Shared Resources</h2> <br /> <p>Many of you have requested more control over the number of builds which can run concurrently for some configurations. This may be to restrict the number of times a particular tool is run due to a license, memory or processor limit, or to prevent concurrency issues with multiple build stages simultaneously writing to the same file, folder or network resource.<br /> You can now allocate quotas to named Shared Resources and specify that builds and stages must acquire a lock on the Shared Resource before running. If all locks are allocated, then the build or stage will wait on the queue until a lock is released.</p> <p>Shared resources can be associated with the server or a particular agent in the Administration pages.</p> <p><img alt="Agent shared resources" src="/blogimages/daves/v1.8/AgentSharedResources.png" /></p> <p>Agent shared resources are acquired when selecting an agent to run a stage. Continua will select the agent with the largest available quota of each shared resource.</p> <p><img alt="Stage shared resource locks" src="/blogimages/daves/v1.8/StageSharedResourceLocks.png" /></p> <p>Server shared resources can also be acquired when selecting an agent, or while on the build queue after evaluating configuration conditions.</p> <p><img alt="Shared resource lock configuration condition" src="/blogimages/daves/v1.8/SharedResourceLockCondition.png" /></p> <p>We hope you find that shared resources can provide many different ways to control the build process.</p> <br /> <h2>Requeue Build</h2> <br /> <p>Sometimes a build may fail due to an offline network resource, or some logical error in the stage workflow. Up until now, your only option was to re-run a build for the same branch heads. If any new changesets had been committed to the branch since that build, then you are out of luck.</p> The new Requeue Build button on the Build View page allows you to requeue an existing build using the same changesets, variables and queue options. Any changes to the configuration such as stage actions or repositories are taken into account and used for the new build. <p><img alt="Requeue build button" src="/blogimages/daves/v1.8/RequeueBuildButton.png" /></p> <p>You can also change the priority, comment and variables before requeuing the build.</p> <p><img alt="Requeue build options menu item" src="/blogimages/daves/v1.8/RequeueBuildOptionsMenuItem.png" /></p> <p>Clicking on the &ldquo;Build requeue options&rdquo; menu item will open the Queue Options dialog.</p> <p><img alt="Requeue options dialog" src="/blogimages/daves/v1.8/RequeueOptions.png" /></p> <br /> <h2>Persist Build Variables</h2> <br /> <p>Another common request has been to persist variable values from one build to another build. This may be to keep a count of builds on a particular branch or to flag that some actions have been completed in one build and do not need to be repeated.</p> Continua CI takes a copy of configuration and project variables at the start of each build. These copies are referred to as build variables. Any changes to build variables are normally discarded when the build finishes and cannot be used by other builds. <p><img alt="First tab of Persist Build Variable build event handler dialog " src="/blogimages/daves/v1.8/PersistBuildVariable1.png" /></p> <p>The new Persist Build Variable build event handler allows you to save the value of the build variables when specific events happen in the build timeline. This is stored as the value of the configuration variable. Subsequent builds will then pick up this revised value and use it as the initial value of the build variable.</p> <p><img alt="Second tab of Persist Build Variable build event handler dialog " src="/blogimages/daves/v1.8/PersistBuildVariable2.png" /></p> <p>As Continua CI allows multiple builds to run concurrently, it is important to control when the variables are overwritten. A later build may run faster and finish before a build which started earlier, causing unexpected results. &nbsp;You can optionally state that a variable should not be persisted if the configuration variable has been modified (e.g. by another build) since a specified build event, such the build start.</p> <p><img alt="Options tab of Persist Build Variable build event handler dialog " src="/blogimages/daves/v1.8/PersistBuildVariable3.png" /></p> <p>You can also prevent concurrency issues by using this feature in conjunction with shared resource locks.</p> <br /> <h2>Other New Features</h2> <br /> <ul> <li>You can now set the Variables display order of variable prompts on the Queue Options dialog.</li> <li>We have provided buttons for cloning Triggers, Repositories and Build Event Handlers.</li> <li>Configuration Conditions can now be disabled.</li> <li>All actions which run external processes now have a Timeout (in seconds) setting.</li> <li>We have also added a new <a href="http://cakebuild.net/" target="_blank">Cake </a>build runner action and a new <a href="https://msdn.microsoft.com/en-us/library/jj155796.aspx" target="_blank">VSTest</a> unit testing action.</li> </ul>745Introducing Continua CI Version 1.8 Betahttps://www.finalbuilder.com/resources/blogs/postid/743/introducing-continua-ci-version-18-betaContinua CITue, 08 Mar 2016 10:10:00 GMT<p> <br /> <br /> </p> This version adds several new features which build upon all the improvements and fixes made to version 1.7.1.<br /> <br /> <h2>Dashboard Filtering</h2> <br /> <p>We've added a new filter box to the dashboard so you can quickly find the configuration (or project) that you are looking for as you type. Use the shortcut key F on the dashboard pages to focus on the filter box and start typing.</p> <p><img alt="Dashboard filtering" src="/blogimages/daves/v1.8/DashboardFiltering.png" /></p> <br /> <h2>Shared Resources</h2> <br /> <p>Many of you have requested more control over the number of builds which can run concurrently for some configurations. This may be to restrict the number of times a particular tool is run due to a license, memory or processor limit, or to prevent concurrency issues with multiple build stages simultaneously writing to the same file, folder or network resource.<br /> You can now allocate quotas to named Shared Resources and specify that builds and stages must acquire a lock on the Shared Resource before running. If all locks are allocated, then the build or stage will wait on the queue until a lock is released.</p> <p>Shared resources can be associated with the server or a particular agent in the Administration pages.</p> <p><img alt="Agent shared resources" src="/blogimages/daves/v1.8/AgentSharedResources.png" /></p> <p>Agent shared resources are acquired when selecting an agent to run a stage. Continua will select the agent with the largest available quota of each shared resource.</p> <p><img alt="Stage shared resource locks" src="/blogimages/daves/v1.8/StageSharedResourceLocks.png" /></p> <p>Server shared resources can also be acquired when selecting an agent, or while on the build queue after evaluating configuration conditions.</p> <p><img alt="Shared resource lock configuration condition" src="/blogimages/daves/v1.8/SharedResourceLockCondition.png" /></p> <p>We hope you find that shared resources can provide many different ways to control the build process.</p> <br /> <h2>Requeue Build</h2> <br /> <p>Sometimes a build may fail due to an offline network resource, or some logical error in the stage workflow. Up until now, your only option was to re-run a build for the same branch heads. If any new changesets had been committed to the branch since that build, then you are out of luck.</p> The new Requeue Build button on the Build View page allows you to requeue an existing build using the same changesets, variables and queue options. Any changes to the configuration such as stage actions or repositories are taken into account and used for the new build. <p><img alt="Requeue build button" src="/blogimages/daves/v1.8/RequeueBuildButton.png" /></p> <p>You can also change the priority, comment and variables before requeuing the build.</p> <p><img alt="Requeue build options menu item" src="/blogimages/daves/v1.8/RequeueBuildOptionsMenuItem.png" /></p> <p>Clicking on the &ldquo;Build requeue options&rdquo; menu item will open the Queue Options dialog.</p> <p><img alt="Requeue options dialog" src="/blogimages/daves/v1.8/RequeueOptions.png" /></p> <br /> <h2>Persist Build Variables</h2> <br /> <p>Another common request has been to persist variable values from one build to another build. This may be to keep a count of builds on a particular branch or to flag that some actions have been completed in one build and do not need to be repeated.</p> Continua CI takes a copy of configuration and project variables at the start of each build. These copies are referred to as build variables. Any changes to build variables are normally discarded when the build finishes and cannot be used by other builds. <p><img alt="First tab of Persist Build Variable build event handler dialog " src="/blogimages/daves/v1.8/PersistBuildVariable1.png" /></p> <p>The new Persist Build Variable build event handler allows you to save the value of the build variables when specific events happen in the build timeline. This is stored as the value of the configuration variable. Subsequent builds will then pick up this revised value and use it as the initial value of the build variable.</p> <p><img alt="Second tab of Persist Build Variable build event handler dialog " src="/blogimages/daves/v1.8/PersistBuildVariable2.png" /></p> <p>As Continua CI allows multiple builds to run concurrently, it is important to control when the variables are overwritten. A later build may run faster and finish before a build which started earlier, causing unexpected results. &nbsp;You can optionally state that a variable should not be persisted if the configuration variable has been modified (e.g. by another build) since a specified build event, such the build start.</p> <p><img alt="Options tab of Persist Build Variable build event handler dialog " src="/blogimages/daves/v1.8/PersistBuildVariable3.png" /></p> <p>You can also prevent concurrency issues by using this feature in conjunction with shared resource locks.</p> <br /> <h2>Other New Features</h2> <br /> <ul> <li>You can now set the Variables display order of variable prompts on the Queue Options dialog.</li> <li>We have provided buttons for cloning Triggers, Repositories and Build Event Handlers.</li> <li>Configuration Conditions can now be disabled.</li> <li>We have also added a new <a href="http://cakebuild.net/">Cake </a>build runner action</li> </ul>743