VSoft Technologies Blogs

Signing .rdp files with Signotaur (and surviving the April Windows update)

Fri, 24 Apr 2026 02:27:48 GMT

In this article:

What changed for signed .rdp files in the April 2026 Windows update
Why previously-working signatures now show an orange warning dialog
The registry policy recipient machines need for no dialog at all
How Signotaur signs .rdp files without distributing the certificate
Deploying the policy via Group Policy or PowerShell

If you push Remote Desktop shortcuts out to users, you probably already know about the April 2026 Windows cumulative update (KB5083769 on Windows 11, KB5082200 on Windows 10). If you don't, you might have found out the hard way — via a Monday-morning deluge of "is this a virus?" tickets from users who suddenly have a big orange warning on the RDP file that worked fine last week.

Signotaur 1.2.0.107 ships with .rdp file signing support. Here's what changed in Windows, why it matters, and what the new signing command actually does.

What Microsoft changed

The April cumulative addresses CVE-2026-26151. Two user-visible things came with it:

The Remote Desktop Connection warning dialog was redesigned. It now lists every resource the connection can redirect (drives, printers, clipboard, USB, etc.) with individual checkboxes, and every box is off by default. Users have to opt in to each one on every connection, every time.
The trust criteria for signed .rdp files tightened. Pre-April, a file signed by an untrusted cert got a yellow "Verify the publisher" banner. Post-April, the same file gets an orange "Caution: Unknown remote connection" banner — visually indistinguishable from an unsigned file.

Here's what the new per-launch dialog looks like. For an unsigned file:

And for a file signed by a publisher Windows can verify:

Separately, the first time any user opens an .rdp file after installing the update, Windows shows a one-time educational dialog explaining what .rdp files are and why they can be dangerous:

Once dismissed, it doesn't reappear for that account.

Discussion and lament: r/sysadmin: FYI: Microsoft RDP changes with April cumulative.

This is a genuinely good security change — the old dialog was vague and under-informed users about what was being shared with the remote host. It is also a rather annoying deployment change, because plenty of teams had rdpsign.exe-signed files quietly working for years, and now they don't.

Why people got caught out

The usual story goes like this: a team signed their .rdp file years ago with a self-signed or internal-CA certificate, shipped it to users, and it displayed a friendly yellow "Verify the publisher: Acme Corp" dialog that everyone clicked through. After April's update, the same file suddenly shows an orange "Unknown remote connection" dialog and support gets flooded. One representative example from the Reddit threads: a sysadmin describing exactly this.

The signatures on those files are still cryptographically valid. Windows is just stricter now about what counts as a "verified publisher" for RDP.

The actual recipe (no dialog at all)

Cutting through the noise, here's what recipient machines need for a signed .rdp file to open with no warning dialog at all:

The signing certificate's chain must terminate in a root the client machine trusts. Commercial code-signing certs (DigiCert, Sectigo, etc.) chain to roots Windows already trusts. For an internal CA or self-signed cert, the root has to be imported into Cert:\*\Root on each client.
A Remote Desktop trust policy must be in place that whitelists your signing certificate's SHA-1 thumbprint. This lives at either HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services (machine-wide, requires admin) or HKCU\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services (per-user, no admin required), and needs two values:
- AllowSignedFiles — REG_DWORD, set to 1
- TrustedCertThumbprints — REG_SZ, the SHA-1 thumbprint of your signing cert, uppercase, no spaces (semicolon-separated if you have more than one)

A word of warning about one trap we hit: TrustedCertThumbprints is a whitelist on top of normal chain validation, not a replacement for it. Dropping a self-signed cert's thumbprint into the list without also importing the cert into a trusted root store does nothing. If you've tried this and wondered why it didn't work, that's why.

What Signotaur does when you sign a .rdp file

Unlike rdpsign.exe, Signotaur signs .rdp files against a remote server — the private key stays on the server (or an HSM it's connected to), so no admin workstation or automation host needs a local copy of the signing certificate. This is the main reason to use Signotaur for .rdp signing rather than running rdpsign.exe directly.

The mechanics otherwise match what you'd expect. The client reads the .rdp file, canonicalises the contents, sends only the digest to the server for signing, and writes the signed file back in place. The output is byte-equivalent to rdpsign.exe's — same canonical form, same 12-byte Microsoft wrapper, same detached CMS structure — so recipients see the same Windows Remote Desktop Connection dialog regardless of which tool produced the signature.

RFC 3161 timestamping is supported, and the timestamp is embedded into the CMS in the standard way. But there's a caveat that's easy to miss: mstsc.exe doesn't actually consult the timestamp when validating a .rdp signature — it only checks whether the signing certificate is currently valid at the moment the file is opened. The strongest indirect evidence for this is that Microsoft's own rdpsign.exe has no timestamping support at all — no /tr flag, no timestamp in its output.

The timestamp is still worth having: it's standards-compliant CMS so general-purpose tools can read the signing time, it provides a tamper-evident audit trail, and it future-proofs against any future change in mstsc's behaviour. In practice though, plan to re-sign distributed .rdp files when your signing certificate is renewed, the same way you would for an unsigned-but-distributed file.

Because the registry recipe above is easy to get wrong, Signotaur also prints the recipe for you after signing, pre-filled with the thumbprint of the certificate that just signed the file. It tells you whether the cert is self-signed (in which case you'll also need to deploy the cert itself to recipient trusted-root stores) or chained (in which case you may not).

If you're running sign interactively on Windows, it also offers to apply the policy to HKCU for the current user on the spot — no admin rights needed, no registry editor, no PowerShell snippet to copy-paste. This is mostly useful for confirming the signing works locally before you deploy the policy via GPO. In unattended runs, the prompt is automatically skipped, and you can pass --no-mstsc-guidance (--nmg) to suppress the whole lot if the log gets noisy.

An example

The sign command detects .rdp files automatically based on extension, so there's nothing special to type:

	
			SignotaurTool.exe sign -a  -s  -t  --tr http://timestamp.digicert.com --td SHA256 connect-to-prod.rdp

After a successful sign you'll see something along the lines of:

RDP signing guidance (Windows Remote Desktop Connection compatibility, post-KB5083769):

By default recipients will see a "Verify the publisher" or "Unknown publisher"
warning dialog when opening this signed file in Windows Remote Desktop Connection.
To eliminate the warning, each recipient machine needs:

1. The signing certificate's chain trusted on the machine.
   For commercial certificates chaining to a Windows-preinstalled root, no action
   is required; internal-CA certificates require the CA to be imported into each
   recipient's Trusted Root store.

2. The signing certificate's thumbprint added to the Remote Desktop trust-publisher
   policy:

   Value: AllowSignedFiles (REG_DWORD) = 1
   Value: TrustedCertThumbprints (REG_SZ) = 

Apply AllowSignedFiles + TrustedCertThumbprints to your current user registry now?
Useful for testing this signed .rdp locally. [y/N]:

Answer y and it writes the values to your HKCU — open the file in Remote Desktop Connection, confirm there's no dialog, and you know the signing bit is working before you take the policy near a GPO.

Deploying the policy to the fleet

For anything beyond a couple of machines, Group Policy is the right tool — it's also how you set these same values without touching the registry by hand. In the Group Policy Management Editor:

Computer (or User) Configuration → Administrative Templates → Windows Components → Remote Desktop Services → Remote Desktop Connection Client
Allow .rdp files from valid publishers and user's default .rdp settings → Enabled
Specify SHA1 thumbprints of certificates representing trusted .rdp publishers → Enabled, with your thumbprints in the list (uppercase, semicolon-separated if multiple)

Intune has equivalent settings under the Administrative Templates profile. If you're deploying to a smaller set of machines, or scripting it, a PowerShell snippet that does the per-user version (no admin needed) looks like this:

	
			$p = 'HKCU:\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services'
New-Item -Path $p -Force | Out-Null
Set-ItemProperty -Path $p -Name AllowSignedFiles -Value 1 -Type DWord
Set-ItemProperty -Path $p -Name TrustedCertThumbprints -Value 'YOUR_THUMBPRINT_UPPERCASE' -Type String

Swap HKCU for HKLM for the machine-wide version (and run elevated). The thumbprint must be uppercase with no spaces; use a semicolon to join multiple thumbprints if you have more than one signing cert in rotation.

What you still can't do

A few things worth knowing up front so you don't hunt for them:

The per-redirection checkboxes in the new dialog are not controlled by AllowSignedFiles. Even with a properly trusted and whitelisted signature, users still see the redirection dialog on first use. There's a separate set of policies for which redirections are allowed on the server side; the client-side checkboxes are a user-consent UI rather than a trust decision.
There's a temporary "revert to pre-April dialog" switch at HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services\Client\RedirectionWarningDialogVersion = 1. Don't rely on it — Microsoft has been explicit that it will be removed in a future update.
Windows has no built-in .rdp signature verifier. rdpsign.exe /v is "verbose", not "verify"; in practice the only verifier is Remote Desktop Connection opening the file. Signotaur's verify command will validate the signature and cert chain offline if you want a CI-friendly check.

If you'd rather not think about any of this

The short version: get a code-signing certificate from a public CA that chains to a Windows-preinstalled root, register it with your Signotaur server, sign your .rdp files as part of your distribution workflow, and push the AllowSignedFiles + TrustedCertThumbprints policy out via GPO. After that, signed .rdp files open with no dialog, and the next time Microsoft tightens the rules you've already done the trust-deployment work.

For command-line details and the full signing command reference see the Signotaur documentation. If you run into something specific — especially if you have an internal CA or a less-common PKCS#11 token in play — our support team is happy to help.

Managing Certificate and API Key Expiry with Signotaur

Fri, 13 Mar 2026 03:18:12 GMT

In this article:

Why code signing certificates now expire sooner
How certificate labels remove thumbprints from build scripts
How to rotate and renew API keys safely
How to automate the process in Continua CI

Shorter Certificate Lifetimes Are Coming

Code signing certificates used to last up to three years. As of March 1, 2026, that is no longer the case.

The CA/Browser Forum adopted Ballot CSC-31 on November 17, 2025, reducing the maximum validity period for publicly-trusted code signing certificates from 39 months to 460 days (roughly 15 months). For most teams, that means certificate renewals are now an annual operational task rather than an occasional one.

When a certificate is renewed, it receives a new thumbprint, and build scripts that reference the old thumbprint must be updated. In large CI/CD environments, tracking down every reference can be tedious and error-prone.

Separately, API keys used by CI agents to authenticate with the signing server have their own lifecycle and eventually need to be rotated. Replacing them without interrupting active build pipelines can be challenging.

Signotaur addresses both concerns. Label-based certificate selection decouples build pipelines from individual certificates, while API key rotation with overlap allows credentials to be replaced without downtime. Together they turn what used to be stressful maintenance tasks into routine operations.

Certificate Labels: Automatic Certificate Rotation

The Thumbprint Problem

Traditionally, build scripts reference a code signing certificate by its SHA-1 thumbprint — a 40-character hex string that uniquely identifies the certificate. When you renew, you get a new certificate with a new thumbprint, so every script that signs code needs to be updated. If you sign across multiple CI configurations, that means tracking down and editing each one.

Labels to the Rescue

Signotaur lets you assign a label to each registered certificate — for example, production or nightly. When you request a signing operation by label instead of thumbprint, Signotaur automatically selects the certificate with the latest expiry date among all enabled, non-expired certificates that share that label. This ensures that newly issued certificates automatically take precedence.

The rotation workflow is straightforward:

Register your renewed certificate and assign it the same label as the outgoing one.
Builds automatically pick up the new certificate — no script changes required.
The old certificate expires naturally; remove it from Signotaur when you're ready.

Expiration Monitoring

Signotaur's background monitoring service checks certificate expiration daily and sends email notifications to administrators at configurable intervals. The defaults are 30, 14, 7, 3, and 1 day(s) before expiry, with an additional alert if a certificate has already expired. You can customise these warning thresholds in the server configuration.

During a rotation window — when both the old and new certificates are registered with the same label — the admin UI highlights the duplicate label so you can confirm the overlap is intentional.

API Key Rotation with Overlap

Why Rotate Keys?

API keys grant access to your signing service. If a key is leaked or reused indefinitely, it becomes a long-lived security risk. Regular rotation limits the impact if a key is compromised, satisfies compliance requirements, and handles natural key expiry gracefully. But naive rotation — revoke old, issue new — creates a window where CI agents holding the old key start failing.

How Signotaur Handles It

The rotate-key command creates a new API key while keeping the old one valid for a configurable overlap period of 0–30 days (default: 7 days). During this window, both keys are accepted, allowing CI agents and environments to transition without interruption.

Once the overlap period expires, the old key is automatically revoked. If you prefer to manage revocation manually, pass --no-auto-revoke and the old key will remain valid until you explicitly revoke it.

Safety Guardrails

Role required — users must be assigned the ApiKeyRotator role before they can rotate keys. An administrator can assign this role in the Signotaur admin UI.
Rate limiting — a maximum of 5 rotations per user per 24-hour period prevents accidental key sprawl.
Linear chain only — a key that has already been rotated cannot be rotated again; you must use the successor key. This prevents branching into multiple active key chains.
Proactive expiry checking — the sign command accepts a --fail-if-expiring-within flag (e.g. 14d). If the API key used for signing will expire within that window, the command fails immediately with a clear message — catching the problem in CI before it causes a real signing failure.

Conditional Rotation

The --if-expiring-within flag on rotate-key makes rotation idempotent: the key is only rotated if it expires within the specified duration (e.g. 14d, 1w, 24h). This makes it safe to run rotate-key on every CI build or on a nightly schedule without generating unnecessary keys.

Example scheduled rotation command:

signotaur rotate-key --if-expiring-within 14d

Key Renewal

By default, a rotated key inherits the old key's expiry date. If the old key had 30 days of validity remaining, the new key also expires in 30 days. Over time, repeatedly rotating a key gradually shortens its remaining lifetime — which can be a problem for long-running automation.

The --renew flag gives the new key a fresh validity period starting from the rotation date, rather than inheriting whatever time was left on the old key. Without a value, it uses the server default of 90 days; with a value (e.g. --renew 180d), it sets a specific duration up to a maximum of 365 days.

This pairs naturally with conditional rotation — check whether the key is approaching expiry, and if so, rotate and renew in a single command:

signotaur rotate-key --if-expiring-within 14d --renew

If the renewal duration would be shorter than the old key's remaining validity (for example renewing for 30 days when 60 days remain), the command fails as a safety measure. Pass --allow-validity-reduction to override this check when the shorter duration is intentional.

Putting It Together in Continua CI

Continua CI ships dedicated Signotaur actions for both signing and key rotation, so you can wire up the entire workflow without writing any custom scripts. Before configuring those actions, you'll need to assign a label to your certificates in the Signotaur admin UI.

Assigning a Label to Your Certificates

In the Signotaur admin UI, navigate to Certificates and either add a new certificate or edit an existing one. In the certificate dialog you'll find two name fields:

Alias — a unique display name for the certificate (auto-generated by default, e.g. "CodeSigningCert (2027-12-31)").
Label — an optional identifier used for CLI and CI/CD certificate selection, e.g. production. Multiple certificates can share the same label. Matching is case-insensitive, so Production and production resolve the same way.

When you renew a certificate, register the new one and assign it the same label as the outgoing certificate. The admin UI will show an orange notice confirming that multiple active certificates share the label and that the one with the latest expiry will be selected automatically. No build script changes needed.

For full details, see the Certificates documentation.

Setting Up the Rotate Keys Action

Add the Signotaur Rotate Keys action to a stage — either in a scheduled maintenance configuration or directly in your build configuration. With the If Expiring Within condition the action only rotates the key when expiry is approaching, so it can safely run on every build.

The action is organised into several tabs:

Signotaur Rotate Keys tab — configure Overlap Days (how long the old key remains valid alongside the new one, default 7) and whether to Disable auto-revocation if you prefer to revoke old keys manually.

Server tab — set the Server URL for your Signotaur instance and the API Key Source. Select Value (from variable) and reference a server variable (e.g. %SignotaurAPIKey%) to avoid hard-coding keys in the action.

Conditions tab — set If Expiring Within to make the rotation conditional (e.g. 14d to only rotate if expiry is within two weeks). Enable Ignore already rotated error to safely run the action in concurrent CI builds without failure if the key was already rotated by another build.

Renewal tab — enable the Renew checkbox to give the new key a fresh validity period instead of inheriting the old key's remaining lifetime. Set a specific Renew Duration (e.g. 90d), or leave it blank to use the server default. Check Allow validity reduction if the renewal duration may be shorter than the old key's remaining validity.

Output tab — choose Set Variable as the output destination and select a Server variable to store the new key. This way your sign actions automatically pick up the rotated key without any manual updates.

Full documentation: Signotaur Rotate Keys Action

Setting Up the Sign Action with Label Selection

Add the Signotaur Sign action to your build stage, after compilation and before packaging.

Key fields:

Files to Sign — glob patterns for your build output, e.g. **/*.exe, **/*.dll.
Certificate Selection Mode — set to Label.
Label — your shared label, e.g. production. Signotaur will automatically select the certificate with the latest expiry date.
API Key Source — point to the server variable populated by the Rotate Keys action, or a securely stored key.
Fail If Expiring Within — e.g. 14d. If the API key expires within two weeks, the action fails early so you can rotate before it becomes a blocker.
Timestamp Server & Digest — configure your preferred timestamping authority and hash algorithm.

Full documentation: Signotaur Sign Action

Typical Rotation Workflow

In practice, certificate and API key rotation can be handled with a small amount of scheduled automation in your CI system.

Register renewed certificates in Signotaur and assign them the same label as the previous certificate.
CI builds continue using the label, automatically selecting the certificate with the latest expiry date.
Schedule periodic API key checks using the rotate-key --if-expiring-within option.
The new key is written to a CI server variable, allowing build pipelines to automatically use the rotated key.
Use --renew so rotated keys receive a fresh validity period rather than inheriting the old key's remaining lifetime.
Retire old certificates and keys once they are no longer needed.

Wrapping Up

With code signing certificate lifetimes dropping to roughly 15 months, the days of "set it and forget it" certificate management are over. Signotaur's label-based signing means your CI/CD pipelines don't need to know — or care — which specific certificate is current. And overlap-based API key rotation ensures you can swap credentials without any signing downtime.

These features are already available in Signotaur. If you're evaluating it, check out the Signotaur documentation to get started.

Using YAML with Delphi - VSoft.YAML

Tue, 16 Sep 2025 13:21:38 GMT

Introduction

Almost every modern Delphi application needs a way to store structured data: configuration settings, runtime options, or even project metadata. For decades, developers have relied on INI files for simple key/value storage, XML for deeply structured documents, or JSON as a lightweight alternative.

But there’s another option that has grown into the de facto standard in DevOps and configuration management: YAML.

This article introduces VSoft.YAML - an open-source Delphi library that makes working with YAML straightforward. We’ll cover why you might choose YAML over INI, XML, or JSON, then dive into practical Delphi code examples for parsing, editing, and writing YAML.

Why YAML?

Human-Readability

YAML is designed for humans first. Compare the following formats describing a database configuration:

YAML

database:
  host: localhost
  port: 5432
  user: admin
  password: secret

JSON

{
    "database": {
    "host": "localhost",
    "port": 5432,
    "user": "admin",
    "password": "secret"
    }
}

XML


    localhost
    5432
    admin
    secret

The YAML version is the least noisy, especially when configurations get larger.

Expressiveness

Supports mappings (key/value), sequences (lists), and scalars (strings, numbers, booleans).
Handles deeply nested structures naturally.
More flexible than INI files, which are limited to sections and key/value pairs.

Adoption

YAML is everywhere, especially in DevOps.

Introducing VSoft.YAML

VSoft.YAML is a pure Delphi implementation of a YAML 1.2 parser and emitter:

No external dependencies — just add the source to your project.
Round-trip capable — parse, modify, and write YAML without losing formatting.
Supports multiple documents in a single YAML stream.
Supports reading/writing JSON
Extensive JSONPath support

Getting Started

Installation

Clone the repo or add it as a Git submodule:

git clone https://github.com/VSoftTechnologies/VSoft.YAML.git

Then add the VSoft.YAML\Source folder to your Delphi project’s search path.

Parsing YAML

The most common scenario is loading YAML from a string or file.

uses
  VSoft.YAML;

var
  Doc: IYAMLDocument;
  YamlText: string;
begin
  YamlText :=
    'database:' + sLineBreak +
    ' host: localhost' + sLineBreak +
    ' port: 5432' + sLineBreak +
    ' user: admin' + sLineBreak +
    ' password: secret';

    Doc := TYAML.LoadFromString(YamlText);
    Writeln('Database host: ', Doc.Root.Values['database'].Values['host'].AsString);
    Writeln('Database port: ', Doc.Root.Values['database'].Values['port'].AsInteger);
end;

To load from a file, just call TYAML.LoadFromFile

Traversing Sequences

YAML supports lists (called sequences).

Example YAML

servers:
- name: web01
  ip: 10.0.0.1
- name: web02
  ip: 10.0.0.2

Delphi Code

var
    doc : IYAMLDocument;
    servers: IYAMLSequence;
    server : IYAMLMapping; 
    i: integer;
begin
    doc := TYAML.LoadFromFile('c:\test\config.yaml');
    servers := Doc.Root.Values['servers'].AsSequence;
    for i := 0 to servers.Count - 1 do
    begin
        server := Servers.Items[I].AsMapping;
        Writeln(Format('%s (%s)',[server.Values['name'].AsString, Server.Values['ip'].AsString]));
    end;
end;

Modifying YAML

Once parsed, you can update values and write them back out:

var
    root: IYAMLMapping;
begin
    root := Doc.Root.AsMapping;
    root.AddOrSetValue('environment', 'production');
    TYAML.WriteToFile(doc, 'c:\test\config.yaml')
end;

Practical Use Cases for Delphi Developers

Configuration files — ship your app with config.yaml instead of INI or XML.
DevOps integration — generate or consume YAML configs for devops tools.
Structured data — when you need nested lists or dictionaries without verbosity.
Collaboration — YAML files are easy for non-developers to read and edit.

Tips and Best Practices

Use JSON when machine-only consumption is expected.
Use YAML when humans will edit the config.

YAML Rules to be aware of

Even though YAML is designed to be human-friendly, it comes with its own quirks. Here are a few issues you might run into when using YAML with Delphi:

1. Indentation Rules

YAML is indentation-sensitive.
Spaces only — tabs are not allowed for indentation.
A single misplaced space can completely change the structure.

valid:
  key1: value
  key2: value

invalid:
  key1: value
   key1: value # wrong indentation

2. Duplicate Keys

Unlike JSON, YAML technically allows duplicate keys in mappings, but this can lead to unexpected behavior. YAML Parsers will typically keep only the last occurrence.

settings:
  timeout: 10
  timeout: 30 # overwrites the first one

VSoft.YAML has a parser option to control the handling of duplicate keys

options := TYAML.CreateParserOptions;
options.DuplicateKeyBehavior := TYAMLDuplicateKeyBehavior.dkError; // default is dkOverwrite
doc := TYAML.LoadFromFile(fileName, options);

Best practice: avoid duplicates in configs, and add validation in your app if needed.

3. Scalars and Quoting

YAML is permissive about quoting strings, which can sometimes surprise you:

path: C:\Temp\file.txt # might be misinterpreted due to backslashes

Safer to quote unusual strings:

path: "C:\Temp\file.txt"

4. Boolean Values

YAML recognizes a wide set of values as booleans: yes/no, true/false, on/off.

feature_enabled: yes # interpreted as boolean true

If you actually mean the literal string "yes", you must quote it:

feature_enabled: "yes"

5. Comments

While YAML supports # comments, they are not part of the data model and usually not preserved

VSoft.YAML does attempt to preserve comments. This is non standard. Generally, comments before mappings (objects) and sequences(arrays) are preserved, comments after scalar values may be preserved.

Blank lines will not be preserved.

6. Large or Complex Documents

For deeply nested structures, readability can drop.
Split configs into multiple YAML files if possible.
Use ---(start) and ...(end) to separate documents in a single file if needed.

7. Type Guessing

By default, YAML interprets unquoted values:

42 → integer
3.14 → float
3.1.4 → string
true → boolean

If you want to enforce string semantics, quote the value:

port: "5432"

8. CRLF vs LF Line Endings

VSoft.YAML is tolerant of Windows vs Unix line endings, but if your configs are shared across systems, normalize line endings in your source control to avoid “it works on my machine” issues.

Conclusion

YAML offers developers a unique combination of human readability, expressiveness, and versatility. Unlike INI files, it can represent complex nested structures; unlike XML, it remains clean and concise; and unlike JSON, it's easier for humans to read and maintain.

The full list of features support by VSoft.YAML is available here.

DPM Package Manager - Progress update

Wed, 24 Feb 2021 10:46:15 GMT

In December 2019, I blogged about a package manager for Delphi that I am working on. This post is a progress update that shows where it's at and what's left to do to get to v1.

DPM Recap

For those not familiar with what I am trying to achieve here, I highly recommend reading my original Delphi Package Manager RFC post. In that post I detailed my ideas, and some of the challenges that Delphi presents when compared to other development environments.

In December 2019, the bare bones of DPM were there. We had a command line tool and that was it. We were able to create packages, install packages (and their dependencies) and restore them (restore ensures all referenced packages are present). Oh and we could list the available packages in our package feed (a folder).

IDE Integration

In the last 13 months there were around 175 commits to the DPM repository. In that time I have added an IDE plugin (that works in Delphi XE2 to 10.4). This involved the creation of several custom controls (I wasn't able to bend any existing ones to work how I wanted it to).

In addition to the work in the project repository, I also published several useful libraries that I needed for this project. DPM is now bootstrapped, to build DPM you need DPM, as it requires several libraries that are referenced as dpm packages.

In Nov 2020 I published the first alpha release that included an installer (code signed by VSoft Technologies) for installing both the command line tool and the IDE plugin (single installer, you can choose which IDE versions to install for). The installer allows you to install for the current user, or for all users (requires elevation to install).

I also did a zoom presentation about DPM to the Melbourne chapter of the Australian Delphi Users Group - a recording of that (long) presentation can be found here.

Adding IDE support for DPM was a massive undertaking. I had very little experience in developing Delphi IDE plugins (using the tools api) - and there were lots of subtle changes between delphi versions, getting things working correctly in 12 versions of Delphi was not easy. In particular, with the later versions of Delphi IDE that use VCL themes, getting things to look right (ie like a native part of the IDE) was a challenge.

The above image shows the installed packages for one of the projects in the project group, you get to this view by right clicking on the project node, or the DPM Packages node in the Project tree.

Note the view only shows the directly installed packages, not the transient dependencies - those you can see in the project tree under the DPM Packages node.

Before you can use DPM in the IDE, you need to configure a package source (a folder where your package files will live)

This can be done fron the command line

dpm sources add -name=local -source=path to the folder you created

Or from the IDE Settings

Compile during install

The most recent updates added support for compiling packages during first install. Packages need to declare how to build in their dspec file, and dpm will use that and call msbuild to compile the packages if needed. DPM also records a bill of materials file (package.bom) in the package cache so that it can tell whether the package needs to be recompiled or not.

On first install, packages that are being compiled during the install process will take a little longer, but on subsequent installs or restores, the process is almost instant (a few ms).

Prior to adding this feature, building dpm on our Continua CI build agents took 13 minutes, much of which was taken up with compiling the dpm packages that it references (in particular, earlier versions of Delphi were very slow with spring4d). Since updating dpm on our agents with the new version, the entire build process for DPM (console app and 12 versions of the IDE plugin and the installer) takes less than 2 minutes.

Missing features

Project group support

When installing packages, the dependency resolution code does not know about other projects in the project group, or what packages and versions they reference. This will be a problem for packages that include design time components that need to be loaded - the IDE can only load 1 version of a design time package. This is what I am currently working on.

Design time packages

DPM does not currently install design time packages into the IDE. This is dependent on project group support, so it's next on the list after project group support.

Package Updates

The ability to detect when package updates are available and make it easy to install those updates. There's an Updates tab in the IDE but it's non functional at this time.

Package Repository

In it's current state, DPM only supports folder based package feeds. This works fine, but it does have some limitations

Limted search abilities - limted to searching on the package filenames.
You have to download packages to a folder.
Package Authors have to host the package files somewhere (mine are under releases on their github projects).

I have made a start on the Package Repository, but not a lot of progress since I'm focusing on the client site right now.

Q & A

Is it usable?

In it's current state, it's only usable for non visual libraries. As I mentioned, the DPM projects all use DPM themselves, and we have DPM actions in FinalBuilder for running the Pack and Restore commands.

If you use any of my open source libraries like DUnitX, Delphi Mocks etc, I have created packages for all of those libraries, and also created mirror projects (just for hosting the package files) for some other popular libraries like Spring4D.

I would encourage library authors in particular to take a look and provide feedback.

Where can we find it?

DPM is an open source project on GitHub, the installer can be found under Releases (under each release, there is an Assets dropdown section).

What versions of Delphi does it support?

Delphi XE2 to 10.4.2 - note that we compile with the latest updates installed for each compiler version.

Why is it taking so long?

Yes, someone asked that recently! This is a side project, free and open source. My primary focus is on running my business and working on our products (that keeps the lights on).

Can we sponsor the project?

Not right now, however it's something I'll look at in the future.

Can we help?

Absolutely. Fork the project on GitHub and clone it to your dev machine and spend some time getting to know the source code. Before making any pull requests, create an issue on github to discuss your ideas and make sure we on the same wavelength!

Advice for Delphi library authors

Wed, 24 Feb 2021 00:57:00 GMT

We use many third-party Delphi libraries to build FinalBuilder and Automise, and that brings plenty of issues when upgrading compiler versions. I've been using Delphi since 1995, both as a developer and as a component vendor, I have learned a thing or two about creating libraries that I would like to share. These are all ideas that make life easier for users, and make it easy to migrate from one version of Delphi to another.

There's no hard and fast rules on how Delphi Libraries are supposed to be structured, these are just my preferences and things I have learned over the years. Hopefully this will help new and existing library authors.

Folder Structure

Keep the Source and the Packages in separate folders, this makes it easier to find the correct packages to compile, e.g :

\Source
\Packages
\Demos

Under Packages, create a folder for each compiler version your library supports, e.g:

\Packages\Rad Studio XE8
\Packages\Rad Studio 10.0
\Packages\Rad Studio 10.1

Package Names

Please, do not put the Delphi version in the package project names.

Bad!!!

MyProjectRun_D10_4.dproj
MyProjectDesign270.dproj

Good

MyProjectRun.dproj
MyProjectR.dproj
MyProjectDesign.dproj
MyProjectD.dproj

Why not put the compiler version in the package project name you might ask? Well the answer is that it makes upgrading compiler versions a major pain for users who link their projects with Runtime Packages (yes, that includes us).

The reason is that when you compile a package, it creates a packagename.dcp file and that is what your project references. So, if your package name is MyPackageRun_D10_4 then that is what will be added to projects that use it.

package MyOwnPackage;
//...
requires
  rtl,
  vcl,
  MyPackageRun_D10_4,
  AnotherPackage_Sydney,
  YetAnotherPackage_D104,
//  ...

When Delphi 10.5 comes out, guess what the user has to do to upgrade their projects.... Yep, replace that all those package references with 10.5 versions (and the multitude of suffixes). Multiply that by a number of projects and a number of libraries (each with potentially multiple runtime packages) and you can see why this might be a pain.

Now you might say, but we don't want 15 versions of MyPackageRun.bpl laying about on users machines, and you would be right. The solution to this is a feature that has been around since Delphi 6 (2001) - LIBSUFFIX.

Setting LIBSUFFIX (on the Description section of project settings) will append the specified suffix to the BPL file name. So a suffix of _D10_4 will result in a package :

MyPackageRun_D10_4.bpl

however, the DCP file will still be generated as :

MyPackageRun.dcp

Remember it's the dcp file that our projects reference (for linking) - so by keeping the dcp file the same for all delphi versions, upgrading to a new compiler version just got a whole lot easier!

So when Delphi 10.5 comes out in the future, all I need to do is install the packages, no changes to my projects.

Update : Someone pointed out that Delphi 10.4.1 support LIBSUFFIX $(Auto) - this will use the Delphi defined PackageVersion - which for 10.4 is 270. This is a nice addition as it makes upgrading the package projects simpler. Of course if you don't like the PackageVersion suffix and use a custom one, then this is not for you.

Use Explicit rebuild, not Rebuild as needed

Have you ever encountered the error

E2466 Never-build package 'XXX' requires always-build package 'YYY'

What this means is, a package, set to Expicit rebuild, references another package, set to 'Rebuild as needed', and it's a pain in the proverbial. Rebuild as needed is also referred to as Implicit Build - in dpk's you will see it as

{$IMPLICITBUILD ON}

If that "Rebuild as needed" package is not part of your project group, guess what, you get to waste time closing and opening projects trying to get it to compile.

I'm sure someone will correct me on this, but I cannot see a good reason to have "Rebuild as needed" set. I suspect this is a hangover from before the Delphi IDE allowed you to specify Project Dependencies and it slows down builds.

Use Search Paths for includes

I often see includes with either hard coded paths, or relative paths like this :

{$I '..\..\MyDefines.inc'}

That's great, if the installer delivers the files in the right place - but they often don't - I hit this issue today, where the package just would not compile. I eventually figured out that the relative path was wrong.

There's a simple fix for this, and that is to remove the path in the $I statement, and use the Project Search Paths feature instead.

I have also seen libraries where there are mulitple copies of the include file and they are slightly different!

Mark packages as Runtime only or Designtime only

Some libraries have their packages marked as "Runtime and Designtime" (the default) - the impact of this is only minor, but it's a pet peeve of mine. The Delphi IDE (in recent versions at least) provides a nice indication of whether packages are runtime or designtime in the project tree, and for designtime packages, whether they are installed.

This makes it simple for me to determine which ones need to be installed or not.

Not Installed

Installed

Summing up

One of the major reasons people do not upgrade Delphi versions is because it's too hard to deal with the third party libraries and all the changes required just to get to the point of compiling. That eventually results in a lack of Delphi sales which results in a lack of investment in Delphi which feeds back into.... well you get the idea ;)

Making third party libraries easier to work with in Delphi has been a bit of a crusade for me, I've been working on this for a while now, and I'm getting closer to a solution - DPM - A package manager for Delphi - if you are a library author, I encourage you to take a look. For examples on how to create a package spec (dspec) take a look at our open source projects https://github.com/vsoftTechnologies/

Introducing DPM - a Package Manager for Delphi

Thu, 12 Dec 2019 09:41:00 GMT

Back in Feb 2019, I blogged about the need for a Package Manager for Delphi. The blog post garnered lots of mostly useful feedback and encouragement, but until recently I could never find a solid block of time to work on it. Over the last few weeks I've been working hard to get it to an mvp stage.

DPM is an open source package/library manager for Delphi XE2 or later. It is heavily influenced by Nuget, so the cli, docs etc will seem very familiar to nuget users. Delphi’s development environment is quite different from .net, and has different challenges to overcome, so whilst I drew heavily on nuget, DPM is not identical to nuget. I also took a close look at many other package managers for other development eco systems.

What is a Package Manager

A package manager provides a standard for developers to share and consume code. Authors create packages that other developers can consume. The package manager provides a simple way to automate the installation, upgrading or removal of packages. This streamlines the development process, allowing developers to get up and running on a project quickly, without needing to understand the (usually adhoc) way the project or organization has structured their third party libraries. This also translates into simpler build/CI processes, with less ‘compiles on my machine’ style issues.

Who and Why

DPM’s initial developer is Vincent Parrett (author of DUnitX, FinalBuilder, Continua CI etc). Why is discussed in this blog post.

DPM Status

DPM is still in development, so not all functionality is ready yet. At this time, it's at the stage where we I would encourage library authors to take a look and play with it and provide feedback (and perhaps get involved in the development). It's very much at a minimum viable product stage. Potential users are of course welcome to look at it and provide feedback, it's just that, well, there are no packages for it yet (there's some test packages in the repo, and I'll be creating ones for my open source libraries). .

What works

Creating packages
Pushing packages to a package source.
Installing packages, including dependencies
Restoring packages, including dependencies.

How do I use it

The documentation is at https://docs.delphi.dev

See the getting started guide.

The command line documentation can be found here.

The Source is on GitHub https://github.com/DelphiPackageManager/DPM

Is DPM integrated into the Delphi IDE

Not yet but it is planned. If you are a wiz with the open tools api and want to contribute then let us know.

Is there a central package source

Not yet but it is planned. At the moment, only local folder based sources are supported. The client code architecture has a provision for http based sources in the future, however right now we are focused on nailing down the package format, dependency resolution, installation, updating packages etc.

Is my old version of delphi supported

Maybe, see here for supported compiler versions. All target platforms for supported compiler versions are supported.

What about C++ Builder or FPC

see here

Does it support design time components

Not yet, but that is being worked on.

How does it work

See this page

Managing Delphi Version Info with FinalBuilder

Wed, 26 Jun 2019 15:49:26 GMT

In this post, we'll take a look at the various options for managing and updating Version Info in Delphi projects using FinalBuilder.

Windows Version Info Primer

Windows Version Info (ie the version info shown in explorer) is stored in a VERSIONINFO resource inside the executable (exe or dll). These resources are created by defining a .rc file, and compiling with either the windows resource compiler (rc.exe) or Delphi's provided resource compiler (brcc32 or cgrc depending on the delphi version). This results in a .res file, which can be linked into exe at compile time by referencing it in the source code, e.g :

{$R 'myresource.res'}

I highly recommend familiarising yourself with the VERSIONINFO resource type and it's parts.

Delphi IDE Support for Version Info

The Delphi IDE creates a [YourProjectName].res file next to the dpr or dpk when you create a new project. This is where the IDE will place the VERSIONINFO resource when you enable the option to "Include version information in project". When you compile the project in the IDE, if needed the IDE will regenerate this res file with updated version info before it is linked into the executable. For exe's, this resource file also includes the MAINICON resource (the icon shown in explorer).

You do not have to use this feature, you can leave the option turned off and manage the version info yourself, by creating your own resource script (.rc) with a VERSIONINFO structure, and compiling it and referencing the resulting .res file in your source code. You can even just reference the .rc file

{$R 'myresource.res' 'myresource.rc'}

and the IDE will compile the rc file and link in the resulting res file. The caveat to this technique is that the command line compiler (dcc32, dcc64 etc) does not support this.

If your binary doesn't have version info, or has incorrect version info, it's typically because :

1) The version info resource wasn't referenced in the source and wasn't linked in
2) There are duplicate VERSIONINFO resources linked, windows will pick the first one it finds.
3) You set the version info on the wrong IDE configuration (more on this below).

The Delphi IDE, along with the dproj file (which is an msbuild project file), uses a convoluted configuration inheritance mechanism to set project properties, including version info. Many a developer has been caught out by this scheme, setting the version info at the wrong node in the heirachy, resulting in no version info in their executables. There have also been issues with dproj files that have been upgraded through multiple versions of delphi over the years.

Using FinalBuilder

In the development environment, the version info usually doesn't matter too much, but for formal releases it's critical, so it's best to leave setting/updating your version info to your automated build tool or your continuous integration server. In FinalBuilder we have invested a lot of time and energy to making version info work correctly with all the different versions of delphi, dealing with the vagaries and subtle differences with each version (and there are many!).

On the Delphi action in FinalBuilder, the Version Info tab presents the version info in a similar way to old versions of delphi IDE did (a much nice ui that the current version imho). This ui allows you control all the various version info properties (and there are a lot!). Note that these will only take effect if you have the "Regenerate resource" option checked on the Project tab (ie, regenerate yourproject.res).

Note that the Major, Minor, Release and Build fields are number spin edits, and cannot take FinalBuilder variables. That can easily be worked around with some simple scripting, in the BeforeAction script event (javascript):

Another option is to use Property Sets to provide the source of the Version Info. Property sets are especially useful when you have multiple actions that need the same version info, or at least to share the same version numbers. Creating a Property Set is trivial, just drop a PropertySet Define action on your target, before the Delphi Action. In the PropertySet Define action, select Win32 Version Info to manage all version info properties, or Win32 Version Numbers to have the property set just manage the major, minor, release and build numbers.

To set the property set values, add a PropertySet Assign Values action before the Delphi action.

Then in the Delphi action it's a simple task to select the property set in the version info tab

Notice that the version number fields are disabled, since they are now provided by the property set. If you choose the Win32 Version Info property set type, more fields are disabled.

One last thing I should mention, is that along with the Version Info and the MAINICON, the project.res file also typically (well for windows at least) contains the manifest file. I recommend you look at the Resource Compiler tab, where it lets you choose which resource compiler to use (more useful in older versions of delphi, where brcc32 didn't cope with some hicolor icon types) and specify the manifest file. I discussed windows manifest files a few years ago in this blog post.

Delphi 10.3 Rio - Language Changes

Thu, 13 Dec 2018 11:12:39 GMT

Back in December 2016, I posted some ideas for some Delphi language enhancements. That post turned out to be somewhat controversial, I received some rather hostile emails about how I was trying to turn Delphi into C#. That certainly wasn't my intent, but rather to modernize Delphi, in a way that helps me write less, but more maintainable code. Nearly 2 years later, Delphi 10.3 Rio actually implements some of those features.

I'm not going to claim credit for the new language features, and the syntax suggestions I made were pretty obvious ones, but I like to think I perhaps spurred them on a bit ;) My blog post had over 12K views, so there was certainly plenty of interest in new language features, and from what I have seen out there on the interwebs they have for the most part been well received.

So lets take a look at which suggestions made the cut for 10.3 - referencing my original post.

Feature	Implemented	Comments
Local Variable Initialisation	No
Type Inference	Yes!	For inline variables only, Confuses code insight!
Inline variable declaration, with type inference and block scope	Yes, Yes and Yes!	Confuses code insight!
Loop variable inline declaration	Yes!	Confuses code insight!
Shortcut property declaration	No
Interface Helpers	No
Strings (and other non ordinals) in Case Statements	No
Ternary Operator	No
Try/Except/Finally	No
Named Arguments	No
Variable method arguments	No
Lambdas	No
Linq	No	Depends on lambdas and interface helpers.
Async/Await	No
Non reference counted interfaces	No
Attribute Constraints	No
Operator overloading on classes.	No
Improve Generic Constraint	No
Fix IEnumerable	No
Yield return - Iterator blocks	No
Partial classes	No
Allow Multiple Uses clauses	No
Allow non parameterized interfaces to have parameterized methods	No

So, 3 out of 23. To be honest, I was pleasantly surprised when I found out about them, given the pace of language change in the past. I'm hopeful this is just the start of things to come and we get to see Delphi evolve and catch up with other modern programming languages. I have a bunch of other language features I'd like to see, and received lots of suggestions from other users.

We're still using Delphi XE7 for FinalBuilder 8, and I rarely change compiler versions during the life of a major product version. So I'll only get to use the new language features when I get fully stuck into FinalBuilder 9 & Automise 6. I'm in the process of getting Delphi 10.3 compatible versions of all the third party libraries (commercial and open source) - as and long time delphi user will know, that's always more difficult than it should be!

Delphi Package Manager RFC

Mon, 16 Jul 2018 01:58:00 GMT

Delphi/Rad Studio desperately needs a proper package/library/component manager. A package manager provides a standardized way of consuming third party libraries. At the moment, use of third party libraries is very much adhoc, and in many cases this makes it difficult to move projects between machines, or to get a new hire up and running quickly.

Other developement environments, like the .net and javascript eco systems, recognised and solved this problem many years ago. Getting a .net or javascript project up an running, in a new working folder or new machine is trivial.

With Delphi/Rad Studio, it's much harder than it should be. In consulting work, I've made it a point to see how clients were handling third party code, and every client had a different way. The most common technique was... well, best described as adhoc (with perhaps a readme with the list of third party products to install). Getting that code compiling on a CI server was a nightmare.

Existing Package Managers

Embarcadero introduced their GetIt package manager with XE8, and the GetIt infrastructure has certainly has made the installation of RAD Studio itself a lot nicer. But as a package manager for third party libraries, it comes up short in a number of areas.

There is also Delphinus, which is an admirable effort, but hasn't gotten much traction, possibly due to it being strongly tied to github (you really need github account to use it, otherwise you get api rate limiting errors).

Rather than pick apart GetIt or Delphinus, I'd like to outline my ideas for a Delphi package manager. I spend a lot of time working with .net (nuget) and javascript (npm, yarn), so they have very much influenced what I will layout below.

I have resurrected an old project (from 2013) that I shelved when GetIt was announced, and I have spent a good deal of time thinking about package management (not just in Delphi), but I'm sure I haven't thought of everything, I'd love to hear feedback from people interested in contributing to this project, or just potential users.

Project Ideals

These are just some notes that I wrote up when I first started working on this back in 2013, I've tried to whip them into some semblance of order for presentation here, but they are just just rough outline of my ideas.

Open Source

The Project should be Open Source. Of course we should welcome contributions from commercial entities, but the direction of the project will be controlled by the community (ie users). The project will be hosted on GitHub, and contributions will be made through Pull Requests, with contributions being reviewed by the Steering committee (TBA).

Public Package Registry

There will be a public website/package server, where users can browse the available packages, and package authors can upload packages. This will be a second phase of the project, with the initial phase being focused on getting a working client/package architecture, with a local or network share folder for the package source.

The package registry should not be turned into a store. Once a public package registry/server is available, evaluation packages could be be allowed, perhaps by providing a fee (web hosting is not free). Commercial vendors will of course be able to distribute commercial packages directly to their customers, as the package manager will support hosting of packages in a shared network or local directory. Package meta data will include flags to indicate if the packages are commercial, eval or free/open source. Users will be able to decide which package types show up in their searches.

Package Submission

Package submission to the public registry should be a simple process, without filling in and signing and faxing of forms! We will follow the lead of nuget, npm, ruby etc on this. There should be a dispute process for package names, copyright infringement etc. There will also be the ability to assign ownership of a package, for example when project ownership changes.

Package Authors will be able to reserve a package prefix, in order to prevent other authors from infringing on their names or copyrights. For example, Embarcadero might reserve Emb. as their prefix, TMS might reserve TMS. as theirs. (of course I'm hoping to get both on board). The project will provide a dispute resolution process for package prefixes and names.

Delphi specific challenges

Delphi presents a number of challenges when compared to the .net or nodejs/javascript world.

Compatibility

With npm, packages contain source (typically minimized and obfuscated) which is pure javascript. Compatibility is very high.

With Nuget, packages contain compiled (to .NET IL) assemblies. A package might contain a few different versions, that target different the versions of the .net framework. Again, compatibility is pretty good, an assembly compiled against .net 2.0 will work on .net 4.7 (.net core breaks this, but it has a new compatibility model, netstandard).

If we look at Delphi, binary compatibility between Delphi compiler versions is pretty much non existent(yes, I know about 2006/7 etc). The dcu, dcp and bpl files are typically only compatible with the version they were compiled with. They are also only compatible with the platform they were generated for (so you can't share dcu's between 32 and 64 bit windows, or between iOS and Android). So we would need to include binaries for each version of Delphi we want our library to support. This also has major implications for library dependencies. Where as npm and nuget define dependencies as a range of versions, a binary dependency in Delphi would be fixed to that specific version. There is a way to maintain binary compatibility between releases, provided the interfaces do not change, however exactly what the rules are for this is hard to come by, so for now we'll ignore that possibility.

That limits the scope for updating to newer versions of libraries, but that can also be overcome by including the source code in package, and providing on the fly compilation of the library during install. My preference would be for pre-compiled libraries, as that speeds up the build process (of course, since that's an area I have a particular interest in). In Continuous Integration environments, you want to build fast and build often, rebuilding library code with each CI build would be painful (speaking from experience here, 50% of time building FinalBuilder is building the third party libraries).

There's also the consideration of Debug vs Release - so if we are including binaries, compiled for Release would be required, but Debug optional? The size of a package file could be problematic. If the package contains pre-compiled binaries for multiple compiler versions, it could get rather large. So perhaps allow for packages that either support a single compiler version, or multiples? The compilers supported would be exposed in the package metadata, and perhaps also in the package file name. Feedback, ideas around this would be welcome.

Package files would be (like with other package managers), a simple zip file, which include a metadata (xml) file which describes the contents of the package, and folders containing binaries, source, resources etc. Packages will not contain any scripts (ie to build during install) for security reasons (I don't want to be running random scripts). We will need to provide a way to compile during install (using a simple dsl to describe what needs to be done), this still needs a lot of thought (and very much involves dependencies).

Library/Search Paths

Say goodbye to the IDE's Library path. It was great back in 1995, when we had a few third party libraries and a few projects and we just upgraded the projects to deal with library versioning (just get on the latest). It's simply incompatible with the notion of using multiple versions of the same libraries these days.

I rarely change major versions of a library during the lifespan of a major release of my products, I might however take minor updates for bugfixes or performance improvements. The way to deal with this is simply to use the Project Search path. Project A can use version 1 of a library, Project 2 can use version 9, all quite safely (design time components do complicate this).

Where a project targets multiple platforms, installing a package should install for all platforms it supports, but it should be possible for the user to specify which platforms they need the package installed for.

Design time Component Installation

The Rad Studio IDE only allows one version of a design time package to be installed at a time. So when switching projects, which might use different versions of a component library, we would need a system that is aware of component versions, and can uninstall/install components on the fly, as projects are loaded.

I suspect this will be one of the biggest project hurdles to overcome, it will requires someone with very good open tools api knowledge (ie, not me!).

Dependencies

Libraries that depend on other libraries will need to specify those dependencies in a metadata file, such that they can resolved during installation. As I mentioned above, binary compatibility issues make dependency resolution somewhat more complicated, but not insurmountable. The resolution algorithm will need to take into account compiler version and platform. The algorithm will also need to handle when a package is compiled from source, for example, binary only packages should not be allowed to depend on source only packages (to ensure compatibility). If we end up with install time package compilation, then some serious work will be needed on the dependency tree algorithm to work our what else needs to be done during install (ie, do any dependencies need to be recompiled?).

This is certainly more complicated than other platforms, and a significant amount of work to get right (ps, if you think it isn't, you haven't considered all the angles!)

General Considerations

Package Install/Restore

The user should be able to choose from a list packages to install. When installing the package, this would be recorded either in the dproj, or a separate file alongside the drproj. The install process will update the project search paths accordingly. Package meta data would control what gets added to the search paths, my preference would be for 1 folder per package, as that would keep the search path shorter which improves compile times.

When a project is loaded, the dproj (or packages config file) would be checked, and any missing packages restored automatically. This should also handle the situation where a project is loaded in a different IDE version.

Security

We should allow for signing of packages, such that the signatures can be verified by the client(s). Clients should be able to chose whether to only allow signed packages, or allow signed and unsigned, and what to do when signature verification fails. This will allow users certainty in the authenticity and integrity of the package (ie where it comes from and whether it's been modified/tampered with).

Clients

It is envisaged that will be at least 2 clients, a command line tool, and a Rad Studio IDE plugin. Clients will download packages, add those packages to project/config search paths. A local package cache will help with performance, avoiding repetitive package downloads and also reduce disk space demands. The clients will also detect available updates to packages, and package dependency conflicts.

Command line Client

The command like tool will be similar to nuget or npm, which provide the ability to create packages, install or restore missing packages, update packages etc. The tool should allow the specification of compiler versions and platforms, as this is not possible to detect from the dproj alone. This is where the project is currently focused (along with the core package handling functionality).

RAD Studio IDE Client

An IDE plugin client will provide the ability to search for, install, restore, update or remove packages, in a similar manner to the Nuget Visual Studio IDE support (hopefully faster!). This plugin will share the core code with the the command line client (ie, it will not call out to the command line tool). I have not done any work on this yet (help wanted).

Delphi/Rad Studio Version Support

Undecided at the moment. I'm developing with XE7, but it's possible the code will compile with earlier versions, or be made to compile with minor changes.

Summary

Simply put, I want/need a package manager for Delphi, one that works as well as nuget, npm, yarn etc. I'm still fleshing out how this might all work, and I'd love some feedback, suggestions, ideas etc. I'd like to get some people with the right skills 'signed up' to help, particularly people with open tools api expertise.

Get Involved!

I have set up a home for the project on GitHub - The Delphi Package Manager Project - RFC. We'll use issues for discussion, and the wiki to document the specifications as we develop them. I have created a few issues with things that need some dicusssion. I hope to publish the work I have already done on this in the next few days (needs tidying up).

Windows 10 Fall Creators Update

Tue, 17 Oct 2017 09:55:38 GMT

The Windows 10 Fall Creators Update has only been out a few hours, but we're already getting questions about it.

In our limited testing, FinalBuilder 8 and Automise 5 run fine.

I've only been running the Fall Update for a few hours, but so far I have not noticed any issues. The applications I use daily all run fine.

The "Windows 10 Creators Update" (ie the one before the Fall Update - stupid release naming imho) broke the Delphi debugger when using runtime packages. Aparently the issue was caused by a library loader optiimisation, not taking into account that dll's can have multiple import tables. I never did see a full explaination or acknowledgement of the problem from Microsoft.

This only affected the debugger (all native code debuggers, not just Delphi), which would load and unload each dll many times (based on the number of imports, for FinalBuilder's core package, it was in the hundreds). Sometimes the application would launch, only for the debugger to crash, sometimes it would just hang, sometimes the Delphi IDE would get out of memory errors.

For me, this was a big issue, since FinalBuilder and Automise use runtime packages. This affected all versions of Delphi, even the latest 10.2 (Tokyo). Embarcadero did eventually ship an update to 10.2 that mostly resolved the problem (not an easy thing as it involved major linker changes), but that didn't help us as we're using an older version (for reasons I won't go into here!).

So since April 2017, I've been really hamstrung when it comes to debugging. Fortunately we discovered the issue before the Creators Update was installed on our other Delphi development machines (and it's a been a constant battle with windows update nagging to install it ever since) so we were still able debug, just not on my dev machine. Frustrating to say the least.

The good news is that the Fall Update (mostly) fixes the problem. I still see some dlls/packages getting unloaded and reloaded again, but the application launches and I can debug.

As far as windows functionality in the Fall Update goes, well the Task Manager has a new GPU section on the performance tab which is mildly interesting, but since I don't use a Pen, or wear a VR headset while working, I'm not noticing much to get excited about. Hopefully, it's just a lot of bug fixes and performance enhancements, minus the show stoppers!!

Windows Manifest Files

Mon, 28 Aug 2017 12:35:00 GMT

In this post I'm going to look at Windows Manifest Files, what they do, why we need them and how to use them in Delphi and FinalBuilder.

We often get asked questions about UAC prompts, High DPI settings, Windows Themes etc when compiling Delphi & C++Builder projects in FinalBuilder. In this post we'll dissect windows manifest files, and look at how the project settings in Rad Studio interact with the manifest file, and why you should use a custom manifest file.

What is a manifest file and what's it for?

A manifest is an xml file, which is typically embedded as a resource (it can be a separate file, but it's not a good practice and is not recommended) in your native windows executable (x86 or x64). This file has information that tells windows (vista or later) what parts of windows it supports, what permissions it needs (UAC), what common control dependencies it has (Windows Side-by-Side loading).

Without a manifest resource, windows has no idea what permissions your application needs, and will not treat your application kindly. You will find things like common controls (file dialogs etc) look strange, attempts to write to files failing and other general misery. In fact your application may fail to run at all on some systems.

Ok, so now we know we really need a manifest, what does that look like.


    
    FinalBuilder is a GUI-based build automation tool for Windows developers.

The above example is actually the manifest file used in the FinalBuilder IDE. Lets break it down.

AssemblyIdentity

The assemblyIdentity tells windows about your application and what the system architecture requirements are. This should be unique. Note that the type, name and version attributes are required. Typically, the version field just has the major version info, I guess you could update it with the exact version each time, but I've never found the need to do that.

Description

The description field is pretty self explanatory.

Dependency

The dependency section is where you describe the side by side dll dependencies, which for Delphi/C++ Builder applications means windows common controls. This is pretty standard stuff, but it's important because without it, your application will have pretty strange looking file dialogs, if it loads at all.

TrustInfo

The trustinfo section is all about security, it tells windows what sort of permissions your application should be given. The requestedExecutionLevel level attribute has 3 possible values :

asInvoker - requesting no additional permissions. This level requires no additional trust prompts.
highestAvailable - requesting the highest permissions available to the parent process. When a standard user runs the application, it will behave the same as asInvoker, ie no UAC prompt. If an administrator runs the application, they will see a UAC prompt.
requireAdministrator - requesting full administrator permissions. All users will see a UAC prompt, standard users will be required to enter an administrator password.

The uiAccess attribute Indicates whether the application requires access to protected user interface elements. Most of the time, this should be set to false. The typical use case for setting it to true is when creating remote desktop style application (like teamviewer etc). If you do set it to true, your application needs to be signed - see this post for code signing in FinalBuilder(https://www.finalbuilder.com/resources/blogs/code-signing-changes-for-2016).

High DPI support.

I'm not going go into to detail on this, it's a complex issue with major differences between windows versions, and limited High DPI support in Delphi. I will say, think very carefully before you enable this, High DPI support in Delphi depends very much on the version of delphi, and third party control support. Don't just enable High DPI support without serious testing. See the msdn doco link at the bottom of this post.

Compatibility

The compatibility section tells windows what versions of windows your application supports. It enables windows functionality in your application. Manifests without a compatibility section default to Windows Vista level functionality.

One of the areas where the compatibility section is very important is detecting the windows version. On Windows 8.1 and 10, in applications that do not specify compatibility with them, GetVersion and GetVersionEx will return 6.2.0.0 for the windows version, rather than 6.3.* for Windows 8.1 and 10.0.* for Windows 10. So your application will think it's running on Windows 8 (or Server 2012) and quite possibly disable functionality.

Don't use Rad Studio's default or auto generated manifest!

I'll explain why in a minute, but first lets look at delphi's support for manifests. Delphi's manifest support differs quite a lot depending on which version of Delphi you are using. I don't have every version installed to check on.

The earliest version I have installed at the moment is D2010, which just has a cryptically named check box :

This option will include a default manifest in the projectname.res file. No options to change anything about that manifest. The manifest included is woefully inadequate.

Fast forward to XE7 :

Things got slightly better (not sure in which XE? version though) as you can now point Rad Studio at a custom manifest file (and you should!) to be included with your application. The default manifest included is still woefully inadequate.

Fast forward again, to Seattle:

Things look different again here, now you can set the badly named "Enable High DPI" and "Enable Administrator Privileges" options. I say badly named, because that's not what those options do. Checking "Enable High DPI" won't make your application support High DPI, it just tells windows it does (when really, it doesn't unless embarcadero fixed it while I wasn't looking). The same applies to the "Enable Administrator Privileges" - it won't give your application Administrator Privileges, it just tells windows your application needs them to run. Semantics shemantics... but I know this has confused many a developer.

Note that the auto generated manifest uses a template, default_app.manifest, which lives in the bin folder, which is typically under program files, so you might need admin access to modify. It's probably a bad idea to modify it anyway, as it will result in a "works on my machine" moment. This template is different in Seattle and later, as it has some "variables" that get substituted by the IDE when building, this file can't be used when building with FinalBuilder as we have no way to get at those variables.

The manifest included is slightly better than before, but still inadequate.

Berlin & Tokyo manifest options are the same as Seattle, just some layout/styling changes. The auto generated manifests have the same limitations as Seattle.

So I said earlier, "Don't use Rad Studio's default or auto generated manifest", and said that the auto generated manifests are inadequate, here's why : they are simply missing information.

There's no assemblyIdentity element, which according to microsoft is required. There's no description element. The High DPI option just sets the dpiAware element to "True/PM", or not at all. You should use a manifest file that is specific to and reflects your application.

For the versions of Rad Studio that support specifying a custom manifest file, just do that. For versions without custom manifest support, uncheck the "Enable runtime themes" option, and add a resource to your project that includes the manifest :

Example manifest.rc

1 24 "E:\\Source\\app\\myapp.manifest"

Using Manifests in FinalBuilder

FinalBuilder has had support for manifest files for a long time (2007, in FB 5), way before Delphi mentioned the word manifest! On the resource compiler tab of the Delphi and C++Builder actions, there is a field to specify the manifest file.

That's all there is to it, FinalBuilder will add that manifest to the projectname.res file (along with the icon and version info).

One last thing, don't forget to add your custom manifest file to your version control, it's source code after all.

References :

MSDN - Application Manifests

Delphi VersionInfo errors

Thu, 17 Aug 2017 11:44:15 GMT

Over the last year or so, we have seen more and more "bug reports" about compiling Delphi projects with FinalBuilder, in particular, reporting issues with compiling version info resources when using Delphi 10.1 (Berlin) and Delphi 10.2 (Tokyo).

This only happens if you tell the Delphi action in FinalBuilder to load version info from the dproj.

The error typically looks something like this :

or this :

"What's this ModuleName variable?? What's com.embarcadero.$(ModuleName) ? I didn't put that there...."

Or "what's this MSBuildProjectName variable?"

This all stems from Embarcadero's support for non windows platforms (in this case I would suggest andriod). The default values for version info Berlin and later have "$(ModuleName)" in the FileDescription, ProductName and ProgramID fields.

This of course is nonsense for the Windows platform, but some bright new embarcadero hire obviously thought this would be a good idea (well that's what I think, who knows what was behind the decision). The problem is, $(ModuleName) is completely unknown outside of the IDE or MSBuild. So when we compile from the command line (using dcc32, dcc64 etc), if your Delphi Action is set to read the version values from the dproj, then you will get this error.

There are three options to resolve this :

Change the FileDescription field in your delphi project settings to something more meaningful, rather than an ad for Embarcadero, like ummm, oh I know, your product name!
Define the ModuleName variable in FinalBuilder - but this still gets the FileDescription field saying com.embarcadero.YourProductName (hey, it's not a java app!).
Don't load the version info from the dproj, let FinalBuilder handle it completely. After all, you are using FinalBuilder to create your production builds, and version information is part of the release process, not the development process. *This would be my recommended option.*

Delphi Language Enhancements

Tue, 06 Dec 2016 14:07:00 GMT

TL;DR - The Delphi language is very verbose, dated and unattractive to younger developers. Suggestions for improvements below.

The Delphi/Object Pascal language really hasn't changed all that much in the last 20 years. Yes there have been some changes, but they were mostly just tinkering around the edges. Probably the biggest change was the addition of Generics and Anonymous methods. Those two language features alone enabled a raft of libraries that were simply not possible before, for example auto mocking (Delphi Mocks, DSharp), dependency injection and advanced collections (Spring4D).

Some of the features I list below have the potential to spur on the development of other new libraries which can only be a good thing. I have several abandoned projects on my hard drive that were only abandoned because what I wanted to do required language features that just didn't exist in Delphi, or in some cases, the generics implementation fell short of what was needed. Many of these potential features would help reducing verbosity, which helps with maintainability. These days I prefer to write less lines of more expressive code.

I have tried to focus on language enhancements that would have zero impact on existing code, i.e. they are completely optional. I have often seen comments about language features where people don't want the c#/java/whatever features polluting their pure pascal code. My answer to that is, if you don't like it don't use it!! There are many features in Delphi that I don't like, I just don't use them, but that doesn't mean they should be removed. Each to his own and all that. Another argument put forward is "feature x is just syntactic sugar, we don't need it". That's true, in fact we don't need any language if we are ok with writing binary code! If a feature is sugar, and it helps me write less code, and it's easier to read/comprehend/maintain, then I love sugar, load me up with sugar.

Inspiration

Lots of the examples below borrow syntax from other languages, rather than try to invent some contrived "pascalish" syntax. The reality is that most developers these days don't just work with one programming language, they switch between multiple (I use Delphi, C#, Javascript & Typescript on a daily basis, with others thrown in on occasion as needed). Trying to invent a syntax just to be different is pointless, just borrow ideas as needed from other languages (just like they did from delphi in the past!) and get on with it!

I have not listed any functional programing features here, I have yet to spend any real time with functional programming, so won't even attempt to offer suggestions.

I don't have any suggestions for how any of these features would be implemented in the compiler, having not written a real compiler before I know my limitations!

Ok, so lets get to it, these are not listed in any particular order.

Local Variable Initialisation

Allow initialisation of local variables when declaring them, eg :

	procedure Something;
	var
	  x : integer = 99;
	begin
	.........

Benefits : 1 less line of code to write/maintain, per variable, initial value is shown next to the declaration, easier to read.

Type Inference

	var
	  x = 99; //it's an integer
	  s = 'hello world'; //it's a string
	begin
	........

Benefits : less ceremony when declaring variables, still easy to understand.

Inline variable declaration, with type inference and block scope

	procedure Blah;
	begin
      .......
	  var x : TStrings := TStringList.Create; //no type inference
	//or
	  var x := TStringList.Create; // it's a TStringList, no need to declare the type
	  .......
	end;

Inline declared variables should have block scope :

	if test = 0 then
	begin
	  var x := TListString.Create;
	  ....
	end;
	x.Add('bzzzzz'); //Compiler error, x not known here!

Benefits : Declare variables when they are needed, makes it easier to read/maintain as it results in less eye movement, block scope reduces unintended variable reuse.

Loop variable inline declaration

Declare your loop or iteration variable inline ( and they would have loop block scope)

	for var item in collection do
	begin
	  item.Foo;
	end;
	item.Bar; //error item unknown.
	
	for var i : integer := 0 to count do
    ....
	//or
	for var i := 0 to count do //using type inference
    ....

Benefits : Avoid the old variable loop value not available outside the loop error, same benefits as inline/block scope etc.

Shortcut property declaration

Creating properties that don't have getter/setter methods in Delphi is unnecessarily verbose in Delph

	
    type
	  TMyClass = class
	  private
	    FName : string;
	  public
	    property Name : string read FName write FName;
	  end;

All that is really needed is :

	type
	  TMyClass = class
	  public
	    property Name : string;
	  end;

Whilst this might seem the same as declaring a public variable, RTTI would be generated differently if it was a variable rather than a property.

Benefits : Cuts down on boilerplate code.

Interface Helpers

Add interface helpers just like for classes and records. Also, remove the limit of one helper per type per unit (without this the above is not really usable). Have a look at how prevalent extension methods are in C#, Linq is a prime example, it's essentially just a bunch of extension methods.

	type
	  TIDatabaseConnectionHelper = interface helper for IDatabaseConnection
	    function Query : IQuerable;
	  end;

The above is actually a class that extends the interface when it's containing unit is used. This would make implementing something like LINQ possible.

Strings (and other non ordinals) in Case Statements

This has to be the most requested feature by far in the history of delphi imho, sure to make many long time Delphi fans happy.

	case s of
	  'hello' : x := 1;
	  'goodbye' : x := 2;
	  sWorld : x := 3; //sWorld is a string constant
	end;

Case sensitivity could be possibly be dealt with at compile time via CaseSensitive 'helper', (or perhaps an attribute). The above example is case insensitive, the example below is case sensitive :

	case s.CaseSensitive of
	  'hello' : x := 1;
	  'goodbye' : x := 2;
	  sWorld : x := 3; //sWorld is a string constant
	end;
    //or
	case [CaseSensitive]s of
	  'hello' : x := 1;
	  'goodbye' : x := 2;
	  sWorld : x := 3; //sWorld is a string constant
	end;

How about :

	case x.ClassType of
	  TBar : x := 1;
	  TFoo : x := 2;
	end;

Benefits : Simpler, less verbose code.

Ternary Operator

A simpler way of writing :

	If y = 0 then
	  x := 0
	else
	  x := 99;

Syntax : x := bool expr ? true value, false value

	//eg
	x := y = 0 ? 0 : 99;

Benefits : Simpler, more succinct code.

Try/Except/Finally

Probably the equal most requested language feature, allowing for try/except/finally without nesting try blocks.

	try
	...
	except
	..
	finally
	...
	end;

Much cleaner than:

	try
	  try
	    ...
	  except
	    ...
	  end;
	finally
	  ....
	end;

Benefits : Neater, Tidier, Nicer

Named Arguments

Say for example, we have this procedure, with more than one optional parameter :

	procedure TMyClass.DoDomething(const param1 : string; const param2 : ISomething = nil; const param3 : integer = 0);

To call this method, if I want to pass a value for param3, I have to also pass a value for parm2

	x.DoSomething('p1', nil,99);

With named parameters, this could be :

	x.DoSomething('p1', param3 = 99);

Yes, this means I have to type more, but it's much more readable down the track when maintaining the code. I also don't need to lookup the order of the parameters, or provide redundant values for parameters that I want to just use their default values for. I also don't end up adding a bunch of overloaded methods to make the method easier to call.

Benefits : More expressive method calls, less overloads.

Variable method arguments

Steal this from C# (which probably borrowed the idea from c++ varargs ... feature)

	procedure DoSomething(params x : array of integer);

The method can be called passing ether an array param

	procedure TestDoSomething;
	var
	   p : array of integer;
	begin
	...... //(fill out the p array)
	  DoSomething(p);
	  //or
	  DoSomething(1);
	  DoSomething(1,2);
	  DoSomething(1,2,3);
	  ........

Benefits : Flexibilty in how a method is called.

Lambdas

Delphi's anonymous method syntax is just too damned verbose, wouldn't you prefer to write/read this:

	var rate := rates.SingleOrDefault(x => x.Currency = currency.Code);

rather than this :

	var
	  rate : IRate;
	begin
	  rate := rates.SingleOrDefault(function(x : IRate) : boolean;
	                                begin
	                                  result := x.Currency = currency.Code;
	                                end);
	...

Ok, so I did sneak in an inline type inferenced local var in the first example, and both snippets rely on the existence of interface helpers (rates would be IEnumerable ) ;)

Benefits : Less code to write/maintain, the smiles on the faces of developers who switch between delphi and c# all the time!

LINQ!

With Lambdas and multiple type helpers per type per unit, a LINQ like feature would possible. Whilst the Delphi Spring library has had Linq like extensions to IEnumerable for a while, this would formalise the interfaces and provides an implementation used by the collection classes. Providers for XML and Database (eg FireDac) would be possible.

Even a Linq 2 VCL & Linq 2 FMX would be possible. A common scenario, update a bunch of controls based on some state change :

	Self.Controls.Where( x => x.HasPropertyWithValue('ForceUpdate', true) or x.IsType).Do( c => c.Refresh); //or something like that.

This would make it possible to operate on controls on a form, without a) knowing their names, b) having references to them, or c) knowing if the control actually exists.

Being able to do this with something like a linq expression would be a massive improvement.. all that's needed is a linq provider for each control framework. Sorta Like jQuery for the VCL/FMX!

Benefits : Too many to list here!

Caveats : Microsoft have a patent on LINQ - so perhaps this isn't really doable? Perhaps with a slightly different syntax. Or challenge the patent!

Async/Await

Bake the parallel library features into the language/compilers, much like C# and other languages have done over recent years (ala async, await). That makes it possible/easier to write highly scalable servers, over the top of asynchronous I/O. Yes, it's technically possible now, but it's so damned difficult to get right/prove it's right that very few have attempted it. This also make it possible to use the Promise pattern, something along these lines - https://github.com/Real-Serious-Games/C-Sharp-Promise to handle async tasks.

Benfits : Write scalable task oriented server code without messing with low level threading, write responsive non blocking client code.

Caveats : Microsoft have a patent on Async/Await

Non reference counted interfaces

Make it possible to use have interfaces without reference counting. I use interfaces a lot, even on forms and frames, I like to limit what surface is exposed when passing these objects around, but it can be painful when the compiler tries to call _Release on a form that is being destroyed. Obviously, there are ways to deal with this (careful clean up) but it's an easy trap to fall in and very difficult to debug.

Possible syntax :

	[NoRefCount] // << decorate interface with attribute to tell compiler reference counting not required.
	IMyInterface = interface
	....
	end;

There would have to be some limits imposed (using the next feature in this list), for example not allowing use of the attribute on an interface that descends from another interface which is reference counted. It would cause mayhem if passed to a method that takes the base interface (for which the compiler would then try generate addref/release calls).

Benefits : Remove the overhead of reference counting, avoid unwanted side effects from compiler generated _Release calls.

Attribute Constraints

(RSP-13322)

Delphi attributes do not currently have any constraint feature that allows the developer to limit their use, so for example an attribute designed to be applied to a class can currently be applied to a method.

Operator overloading on classes.

Currently operator overloading is only available on records, adding them to classes would make it possible to create some interesting libraries (DSL's even).

I had discussions with Allen Bauer (and others over) many years ago about about this. Memory management was always the stumbling block, with ARC this would not be big issues. Even without ARC, I think delphi people are capable of dealing with memory management requirements, just like we always have.

Edit : I believe this feature is actually available on the ARC enabled compilers.

Improve Generic Constraints

Add additional constraint types to generics, for example :

	type
	  TToken = class
	    .....
	  end;

Benefits : more type safety, less runtime checking code required.

Fix IEnumerable

Not so much a language change rather than a library change, please borrow spring4d's version - much easier to implement in 1 class (delphi's version is impossible to implement in 1 class as it confuses the compiler!).

Yield return - Iterator blocks

The yield keyword in C# basically generates Enumerator implementations at compile time, which allows the developer to avoid writing a bunch of enumerator classes, which are essentially simple state machines. These enumerators should be relatively easy for the compiler to generate.

This a a good example (sourced from this stackoverflow post

	public IEnumerable Read(string sql, Func make, params object[] parms)
	{
	  using (var connection = CreateConnection())
	  {
	    using (var command = CreateCommand(CommandType.Text, sql, connection, parms))
	    {
	      command.CommandTimeout = dataBaseSettings.ReadCommandTimeout;
	      using (var reader = command.ExecuteReader())
	      {
	        while (reader.Read())
	        {
	          yield return make(reader);
	        }
	      }
	    }
      }
	}

The example above will read the records in from the database as they are consumed, rather than reading them all into a collection first, and then returning the collection. This is far more memory efficient when there are a lot of rows returns. In essence, the consumer/caller is "pulling" the records from the database when required, this could for example be pulling messages from a queue.

I guess for delphi, this could be a simple Yield() method( or YieldExit() to more more explicit).

Benefits : less boilerplate enumerator code, lower memory usage when working with large or unbounded datasets or queues etc.

Partial classes

Yes I know, this one is sure to kick up a storm of complaints (I recall an epic thread about this topic on the newsgroups about this years ago). Like everything else, if you don't like it, don't use it. I don't often use this feature in C#, but it is indispensable for one particular scenario, working with generated code. Imagine generating code from an external model or tool for example type libraries, uml tools, database schema, orm, idl etc. The generated code would typically be full of warning comments about how it's generated code and shouldn't be modified. Partial classes get around this, by allowing you to Add code to the generated classes, and the next time they are regenerated, your added code remains intact. Simple, effective.

Benefits : Enables code generation scenarios without requiring base classes.

Allow Multiple Uses clauses

(RSP-13777)

This is something that would make commenting/uncommenting units from the uses clause a lot easier, and also make tooling of the uses clause easier.

	uses
	sharemem, System.SysUtils, System.Classes, vcl.graphics{$ifdef debug},logger{$endif};

The above syntax is easily messed up by the IDE.

	uses sharemem;
	uses System.SysUtils, System.Classes;
	uses vcl.graphics;
    {$IFDEF debug}uses logger;{$endif}

This makes commenting-out and reorganising the library names more convenient. It would also make refactoring and other tooling easier.

Allow non parameterized interfaces to have parameterized methods

(RSP-13725)

	IContainer = interface
	['{2B7B3956-7101-4619-A6DA-C8AF61EE4A81}']
	  function Resolve: T;
	end;

That won't compile, but this code works as expected:

	TContainer = class
	  function Resolve: T;
	end;

This is a limitation I have come across many times when trying to port code from C# to Delphi.

Conclusion

That's 20+ useful, optional and simple to implement (just kidding) language features that I would personally like to see in Delphi (and I would use every one of them). I could have gone on adding a lot more features, but I think these are enough to make my point.

Here's my suggestion to Embarcadero, invest in the language and make up for lost time! While you are at it, sponsor some developers to try porting some interesting and complex open source libraries to delphi, and when they hit road blocks in the language or compiler(and they will), make it work, iterate until baked. I tried porting ReactiveX https://reactivex.io/ to delphi a while back, but hit too many roadblocks in the language and compiler (internal compiler errors, limitations in the generics implementation).

The end result would be a modern, capable language that can handle anything thrown at it.

VSoft.CommandLineParser for Delphi - Updated

Thu, 10 Dec 2015 11:24:20 GMT

A while back I published the VSoft.CommandLineParser library on github, which makes it simple to handle command line options in delphi applications. The first version only did enough to satisfy the needs I had in DUnitX.

In another project I’m working on, I needed a command mode, where each command had a unique set of options, but keeping the ability to have global options. I have tried to implement the command mode in a backwards compatable manner, and so far the only change I had to make to an existing project was adding a const to a parameter.

Adding Commands

Adding commands is quite simple, using TOptionsRegistry.RegisterCommand.

cmd := TOptionsRegistry.RegisterCommand('help','h','get some help','','commandsample help [command]'); 
option := cmd.RegisterUnNamedOption<string>('The command you need help for', 
  procedure(const value : string) 
  begin 
    THelpOptions.HelpCommand := value; 
  end);

Note: this method returns a TCommandDefinition record that you can add options to. The reason for using a record rather than an interface here, is because delphi interfaces do not suport generic methods. Records do, so we use the record type as a wrapper around the ICommandDefinition interface.

The helpstring parameter allows you to specify a longer help message that can be displayed when showing command usage.

Handling Commands

The ICommandLineParseResult interface has a new Command property (string) which is used to determine the selected command. It’s up to you how to actually run the commands.

Showing Usage

The PrintUsage method now has some overloads and has some formatting improvements, and TOptionsRegistry also has new EnumerateCommands and EmumerateCommandOptions methods which make it relatively simple to handle showing usage etc yourself if you want to.

Where is it?

The source with samples is available on GitHub - https://github.com/VSoftTechnologies/VSoft.CommandLineParser

Modifying XML Manifest files with FinalBuilder

Mon, 07 Dec 2015 11:37:47 GMT

This topic is something that I pulled from our support system, it's something we get asked about more that once a year, that is, how do I modify the xml manifest file using FinalBuilder. Typically its the assembly version attribute that users want to modify, so that's what I'll show here, but you can use the same technique to edit other parts of the manifest file.

So lets define our XML Document by adding an XML Document Define Action, and point it at our manifest file.

If you open your manifest file in notpad, you will notice assembly element looks something like this :

<assembly xmlns="urn:schemas-microsoft-com:asm.v1"  manifestVersion="1.0" >

Note the xmlns attribute, this is what causes users problems with the xml actions in FinalBuilder, XML Namespaces. The MXSML Parser is very strict when it comes to namespaces, and it requires that we make use of them when using XPath to select nodes. On the XML Document action, switch to the MSXML Parser tab and in the Extra Namespaces grid, add the following.

What we are doing here is assigning a prefix (x in this case) to the namespace. This prefix will be used in our XPath statements.

Add an Edit XML File action - set the XML File to use an XML Document and select the document we defined with the previous action. Now we need to define the XPath statement to the version attribute that we are going to modify.

And finally, add a Save XML Document Action to save our changes to the file. Note that if you are editing other parts of the manifest file, make sure you add the namespaces and different prefixes, and use this prefixes appropriately in your XPath statements.

Note, all of this could be done in a single Edit XML File action, however, if you want to make more than one modification to the manifest file then it's more efficient to use the xml document define action to avoid loading/parsing/saving the document for each edit.

Delphi Code Coverage with Continua CI

Sun, 08 Nov 2015 12:35:00 GMT

Testing code is something we all do. Whether it be manual usability testing, unit testing, or integration testing, knowing how much of the application is covered by the tests is important. Without knowing what parts of the application are covered, there is no way to know if key features are tested.

When performing unit testing there is an analytical way to determine what parts of the source code are covered by the tests. This is typically call source code coverage. Working with Delphi, one of the tools that performs this task is called DelphiCodeCoverage (open source). It can be located on GitHub (more recent fork) and SourceForge. Under the hood this tool simply marks each line of source code as "hit" when the application calls it at least once. From there it can generate a detailed report giving the overall coverage statistics for the project, as well as the individual lines not hit in the testing.

Code Coverage

What I will go through below is how to setup code coverage on a unit test project, and hook that into a continuous integration process using Continua CI. I will assume that if you require knowledge on how to setup a project on Continua CI you will refer to the Create your First Project wiki page.

The code that I would like to get a code coverage report on is the Core.Card.pas unit. The unit tests for this class are located in the tests folder and have a corresponding name of Core.CardTests.pas. You may have noticed that some of the code paths are not fully covered in my unit tests. This is intentional, and something that we will come to a little later on.

An extra consideration to have with a unit testing project is to make sure it can run under continuous integration. This means that it should run to completion and produce an output file that is able to be imported into the build summary. With this in mind I have created a "CI" configuration on my unit testing project. This conditionally compiles the unit testing project so that it does not wait for user input (something my debug configuration does) and generates an XML output file.

All this code and other related scripts are located in the VSoftTechnologies/DelphiCodeCoverageExample GitHub repository. Feel free to clone it to get a better sense of code coverage and the project structure I am using.

DelphiCodeCoverage

To generate a code coverage report I decided to use DelphiCodeCoverage. The tool has a number of command line options, all of which are spelt out on the GitHub page. Some of the options are a little overwhelming in the effort they require. An example of this is passing a file that contains all the source directories to search for classes to include in the code coverage report. Thankfully there is a wizard supplied with DelphiCodeCoverage that will help generate a batch file containing the correct parameters to pass to DelphiCodeCoverage.

In my project I have placed DelphiCodeCoverage into a sub-folder call "CodeCoverage" and included it into source control. There are two reasons I am doing this;
1. The code coverage executable is now available everywhere the source is pulled to.
2. It simplifies the script I will need for the CI Server.
If your uncomfortable with placing binaries into your source control, this can be altered without affecting the produced report.

Running the code coverage wizard your presented with a page to enter the executable, map file, source, and output directory locations. Below are the settings I have used:

The last option for the wizard allows for making all paths relative. This is exactly what we require to have our generated batch file run on any system, however at the time of writing it does not work correctly. This meant that I had to manually change all paths to a version that was relative to the folder in which DelphiCodeCoverage was located.

dcov_execute.bat before

"I:\Examples\DUnitX_CodeCoverage\CodeCoverage\CodeCoverage.exe" 
-e "I:\Examples\DUnitX_CodeCoverage\Win32\Debug\DUnitX_And_CodeCoverage.exe" 
-m "I:\Examples\DUnitX_CodeCoverage\Win32\Debug\DUnitX_And_CodeCoverage.map" 
-uf dcov_units.lst -spf dcov_paths.lst 
-od "I:\Examples\DUnitX_CodeCoverage\CodeCoverage\Output\" -lt -html

dcov_execute.bat after

"CodeCoverage.exe" 
-e "..\Win32\CI\DUnitX_And_CodeCoverage.exe" 
-m "..\Win32\CI\DUnitX_And_CodeCoverage.map" 
-ife -uf dcov_units.lst -spf dcov_paths.lst -od ".\Output\" -lt -html

Note: Line breaks are only included above for readability. There are none in the resulting batch files.

Another alteration that I have made to the batch file is to include the "-ife" option. The option will include file extensions. This means that it will stop a unit like "Common.Encoding" being 'converted' to "Common". As in my project I have unit called "Core.Cards.pas" this option is required to have it included in generated code coverage report.

Next the relative path change should be applied to the two generated list files dcov_paths.lst and dcov_units.lst. The paths file should be the only one that has path in need of altering to be relative. Both however need to be checked to make sure they contain everything to be covered in the report. If there are source folders missing they need to be added to the dcov_paths.lst file. If there are unit names missing they need to be added to the dcov_units.lst file.

Now that the batch file and list files have been corrected running the dcov_executable.bat should produce summary out similar to that below. Note that the unit test project needs to be compiled as DelphiCodeCoverage runs the unit test executable.

**********************************************************************
*              DUnitX - (c) 2013 Vincent Parrett                     *
*                    vincent@finalbuilder.com                        *
*                                                                    *
*        License - https://www.apache.org/licenses/LICENSE-2.0        *
**********************************************************************
  Fixture : Core
  -------------------------------------------------
     Fixture : Core.CardTests
     -------------------------------------------------
        Fixture : Core.CardTests.TCardTest
        -------------------------------------------------
          Test : Core.CardTests.TCardTest.A_Card_FacingUp_Once_Flipped_Is_Facing_Down
          -------------------------------------------------
          Executing Test : A_Card_FacingUp_Once_Flipped_Is_Facing_Down
            Success.
         Running Fixture Teardown Method : Destroy

         Done testing.
         Tests Found   : 1
         Tests Ignored : 0
         Tests Passed  : 1
         Tests Leaked  : 0
         Tests Failed  : 0
         Tests Errored : 0

Summary:
+-----------+-----------+-----------+
|   Lines   |  Covered  | Covered % |
+-----------+-----------+-----------+
|        15 |        11 |      73 % |
+-----------+-----------+-----------+

Continuous Integration

With the code coverage batch file we are now able to run code coverage on any system, include on a continuous integration system. Our goal with the continuous integration is to have the unit tests built and run each time a set of code is checked into source control. This will allow us to then track if any unit tests fail, and changes in the code coverage.

To achieve this I have create a Continua CI configuration that builds my unit test project, runs the unit tests under code coverage, and then import the unit test results into the build summary.

The FinalBuilder action calls the FinalBuilder project responsible for compiling the DUnitX unit test project. It uses the CI configuration so that the unit tests executable will run to completion, and will produce an NUnit XML results file in the same directory as the executable. It is important to build the unit tests each time as the source code for our project would have changed each time we run the continuous integration. Note that you do not have to use FinalBuilder, you can also use MSBuild to build your DUnitX Project - see Integrating DUnitX Unit Testing with Continua CI.

The execute program action simply runs the code coverage batch file generated above. This batch file will run the unit test project we compiled and log code coverage information as it does. The result will be a summary written out to our build log while also html files written to the report folder we specified in the batch file. It is these html files which we will attach to the continuous build report a little later.

Lastly we want to import the actual unit test results. These are written out by DUnitX as a NUnit compatible XML file which we can import with the "Import NUnit Tests" action. The results from the XML file will be attached to the build report presented by Continua CI.

As all builds for Continua CI are run on agents, and all build reports come from the server, we need to transfer the code coverage report back to the server. This is done through workspace rules on the build stage. In this example DelphiCodeCoverage writes out all html report files to a relative directory of ".\Output\". This means if we run the DelphiCodeCoverage batch file from "Source\Output\CodeCoverage\" the report should appear in "Source\Output\CodeCoverage\Output" (Note that $Source.DelphiCodeCoverage.Path$ was mapped to the \Source\ folder on the agent). Workspace rules use the greater than symbol to signal the files should be copied from the server to the agent, and the less than symbol to copy from the agent to the server. This therefore leaves use the workspace rule of "“/Output/CodeCoverage/ < \Source\CodeCoverage\Output\*.html" to get all code coverage report files back to the server.

Now that the html reports are on the server, we need to show them against the Continua CI build. To achieve this we use the reports section of our Continua CI configuration. The reports section allows us to specify a file to attach to the build as a report to be displayed or offered as a download. In this case we want to display the report summary html file. All reports work from the server point of view, and each build has it own workspace on the server. To this end the report we want to be display would have been copied to "$Workspace$\Output\CodeCoverage\CodeCoverage_summary.html".

The Code Coverage Report

The end report appearing in the report section of the Continua CI build summary.

As shown in the report the example project has some code that is not covered during unit testing. This reduces the overall coverage to 73%. If I had more than one unit each would have their own code coverage summary. In addition I could click on each file and get a line by line report to see what section of the unit is not covered.

Final Notes

It is worth mentioning that code coverage is only one arrow in a software testing quiver. In my example I purposely chose to include code that was not covered. This showed the power of code coverage in picking up where unit testing should potentially be directed to next. I also included code where the unit tests cover the code, however not fully. The code testing Core.Card.Flip only tests one path through the code, not all the possible paths. Currently the test sees if the code works when going from face up to face down, not from face down to face up. Although in this example it might be benign, it shows that other tools are needed to help cover this gap.

Delphi-Mocks Parameter Matchers

Tue, 22 Sep 2015 10:15:11 GMT

We recently updated Delphi Mocks to allow for better parameter matching on Expectations registered with the Mock. This allows the developer to place tighter controls on verifying that a mocked interface/object method is called. Below is a simple example of when the parameter matchers can be used.

procedure TExample_InterfaceImplementTests.Implement_Multiple_Interfaces;
var
  sutProjectSaver : IProjectSaveCheck;
  mockProject : TMock<IProject>;
begin
  //Test that when we check and save a project, and its dirty, we save.
  //CREATE - The project saver under test.
  sutProjectSaver := TProjectSaveCheck.Create;
  //CREATE - Mock project to control our testing. 
  mockProject := TMock<IProject>.Create;
  //SETUP - Mock project will show as dirty and will expect to be saved.  
  mockProject.Setup.WillReturn(true).When.IsDirty;

  //NEW! - Add expectation that the save will be called as dirty is returning true. 
  //       As we don't care about the filename value passed to us we 
  //       allow any string to be passed to report this expectation as met. 
  mockProject.Setup.Expect.Once.When.Save(It(0).IsAny<string>());
    
  //TEST - Visit the mock element to see if our test works.
  sutProjectSaver.Execute(mockProject);
  //VERIFY - Make sure that save was indeed called.
  mockProject.VerifyAll;
end;

Previously the developer writing this test would have to provide the exact filename to be passed to the mocked Save method. As we don't know what the projects filename is going to be (in our example case), we would either have to; 1. Forgo doing this test. 2. Implement a project object to test with. Both of these options are not ideal.

Parameter matchers resolve this situation. It is now simple to either restrict or broaden the parameters passed to mocked methods that will satisfy the expectation defined. To achieve this Delphi-Mocks offers eleven new functions;

function It(const AParamIndx : Integer) : ItRec;
function It0 : ItRec;
function It1 : ItRec;
function It2 : ItRec;
function It3 : ItRec;
function It4 : ItRec;
function It5 : ItRec;
function It6 : ItRec;
function It7 : ItRec;
function It8 : ItRec;
function It9 : ItRec;

The first "function It(const AParamIndx : Integer) : ItRec;" allows the developer to specify the index of the parameter they wish to set for the next expectation setup of a mock method. It(0) will refer to the first parameter, It(1) the second and so forth. Note that the reason for specifying the parameter index is that Delphi's parameter evaluation order is not defined, so we could not rely on the parameters being evaluated in order (which is what we did when we initially wrote this feature). Interestingly, with the 64 bit Delphi compiler, parameter evaluation does appear to happen in order, but we could not be certain this will always be the case.

The other ten functions It0 through to It9 are simply wrappers of the index call passing the index in their name. All these functions return an ItRec. The ItRec has the function structure;

ItRec = record
  var
    ParamIndex : cardinal;
  constructor Create(const AParamIndex : Integer);
  function IsAny<T>() : T ;
  function Matches<T>(const predicate: TPredicate<T>) : T;
  function IsNotNil<T> : T;
  function IsEqualTo<T>(const value : T) : T;
  function IsInRange<T>(const fromValue : T; const toValue : T) : T;
  function IsIn<T>(const values : TArray<T>) : T; overload;
  function IsIn<T>(const values : IEnumerable<T>) : T; overload;
  function IsNotIn<T>(const values : TArray<T>) : T; overload;
  function IsNotIn<T>(const values : IEnumerable<T>) : T; overload;
  {$IFDEF SUPPORTS_REGEX} //XE2 or later
  function IsRegex(const regex : string; const options : TRegExOptions = []) : string;
  {$ENDIF}
end;

Each of the functions creates a different matcher. For example the IsAny<T> will cause the expectation to be met when the parameter passed to the mock is of any value that has the type T. In the example above this type would be a string. You will also notice that each function returns the type T. This is so that each call can be placed within the mock methods call directly. Doing so helps with making sure parameter types match the testing value.

IsEqualTo<T> requires that the parameter matches exactly to the value passed into the IsEqualTo<T>. This could be used to restrict the expectation to a tighter test of the functionality under test.

//Match on the filename being "temp.txt" only.
mockProject.Setup.Expect.Once.When.Save(It(0).IsEqualTo<string>('temp.txt'));
//VERIFY - Make sure that save was indeed called.
mockProject.VerifyAll;

In the future we are looking to provide “And”\”Or” operators. These operators might also live on the ItRec and allow combining with as many other matchers using the same type.

//Match on the filename being "temp.txt" or "temp.doc" only.
mockProject.Setup.Expect.Once.When.Save(
       It(0).Or(It(0).IsEqualTo<string>('temp.txt'), 
                It(0).IsEqualTo<string>('temp.doc'));
//VERIFY - Make sure that save was indeed called.
mockProject.VerifyAll;

There might be a better way to make the resulting code a bit cleaner. It would make the tests easier to read, instead of using regex which is also possible in this case. As a result we believe this would be a good edition to the library.

Feel free to clone the repository from GitHub. If you have some time to spare submit a pull requests or two with your ideas/improvements. We believe this is a great little project worthy of some attention. Let us know what you think of the changes so far.

FinalBuilder 8 Released!

Thu, 20 Aug 2015 19:02:00 GMT

It's very nearly 5 years since FinalBuilder 7 was released. Since it's release we have shipped many official updates, nearly every update including new features or improvements. This program of continuous improvement has worked well, with customers not having to wait for major new versions to arrive to get support for new versions of Visual Studio or Delphi etc, but it has limited our ability to make major changes. So it's time for a new major version of FinalBuilder.

What's new in FinalBuilder 8

IDE Themes

The IDE has two new themes, Dark and Light (yes, imaginatively named!). The IDE defaults to Dark on first run, however you can change the theme in the options quite easily.

Debugger

One of the most asked for features now available in FinalBuilder 8, stepping into included projects. In FinalBuilder 7 and earlier, you could only step over included projects, and wait for them to return. In FinalBuilder 8, you can step into the included project, if it is not already opened the IDE will open the project and switch to it automatically. To make this possible, there are now "Step Into" and "Step Over" functions. The Step into/over now also applies to targets (see below).

Debugger breakpoints now have conditions :

Actionlists renamed to Targets

ActionLists have been renamed to Targets. Targets can now also define dependencies, so you can for example define Clean, Build, Test, and have Test depend on Build. If you execute the Test target, and Build has not already been executed, it will be executed first before Test. Targets can be specified on the command line.

In FinalBuilder 7 and earlier, projects had a Main and an OnFailure (global error handler) actionlist. In FinalBuilder 8, projects just have a Default Target. Older projects will be imported such that the Main and OnFailure Targets are called from the Default Target inside a try/catch block.

Run Target Action

You can now return values from Targets (ie out parameters) .

New Help System

The help has moved online in the form of a wiki. This enables us to do inline help updates without needing to ship new builds. The new help is still being worked on, lots of screenshots are missing etc..

Non Visible Changes

Stepping Engine

The stepping engine was rewritten to enable stepping into included projects, and to enable target dependencies. This, work, together with the new variables architecture is where the bulk of effort/time was spent in the FinalBuilder 8 development cycle.

Variables Architecture

The variables architecture and the expression evaluator were rewritten to resolve several corner case issues that we were not able to resolve in FinalBuilder 7. The expression evaulator has a new parser that will allow us to more easily extend the syntax in the future. The User variable namespace was removed, it caused too many problems with projects not running under other users, not running on the build server etc. Use Project variables instead.

Core Messaging

Changes to the messaging has allowed us to improve the performance of the stepping engine and logging, with much less thread switching. This also improved the IDE performance.

CLR Hosting

The minimum CLR version is now .NET 4.0 (ie FinalBuilder requires .net 4.0 to be installed). FinalBuilder 8 also requires Powershell 3.0 or later.

Code Changes

In addition to the architectural changes, we also spent a lot of time refactoring the code, running static analysis tools over the source, looking for memory leaks, potential bugs etc. One of the results of this is reduced memory usage during a build compared to FB7. The FB8 IDE does use slightly more memory than the FB7 IDE at startup (mostly due to the heavy use of delphi generics), however the runtime memory usage is much lower.A large part of the refactoring involved unit testing (we created a new unit test framework to suite our needs!) and creating a suite of integration tests.

FBCmd

The command line parameters have changed to be more consistent and easier to specify. You can also specify one or more targets to execute (when not specified, the default target is executed).

New Project File Formats

FinalBuilder has used an xml file format since version 1, however a common complaint over the years, has been that it is difficult to diff file versions. FinalBuilder 8 has tackled this in two ways.

A new DSL style project file format (.fbp8) is now the default format, it is very easy to diff.

project
begin
    projectid = {04710B72-066E-46E7-84C7-C04A0D8BFE18}
    target
    begin
        name = Default
        targetid = {E6DE94D6-5484-45E9-965A-DB69885AA5E2}
        rootaction
        begin
            action.group
            begin
                id = {D860420B-DE46-4806-959F-8A92A0C86429}
            end
        end
    end
end

A new xml format (.fbx8), much less verbose than the old format.



    
        {6A717C24-D00F-4983-9FD0-148B2C609634}
        
            Default
            {E6DE94D6-5484-45E9-965A-DB69885AA5E2}
            
                
                    {D860420B-DE46-4806-959F-8A92A0C86429}

Compressed project files (.fbz8) use the dsl format internally (compressed projects are just a zip file with a project.fbp8 inside it).

The default project file encoding is now UTF-8, which is more version control friendly (some version control systems treat utf-16 as binaries).

New Actions

FinalBuilder 8 includes new actions for Chocolatey, Bower, NPM, Gulp, Grunt, Rake, Fake, Mocha, along with Redgate SQL Compare and SQL Data Compare actions.

TFS 2015 XAML builds are also supported, a VSO task will be available soon (will be published on github).

License Key installation

We implemented a new more reliable trial mechanism for FinalBuilder 8, and made it simpler to install license keys. You can log into your account on our website directly in the FinalBuilder IDE and download & install license keys

The trial mechanism also uses license keys, which is more reliable than the mechansim used in earlier versions.

Where's my license key?

If you had an active Software Assurance Subscription for FinalBuilder 7 as of 20th August 2015, FinalBuilder 8 license keys were generated and added to your account today. You should have received an email notification.

How do I purchase an upgrade?

If you had a subscription for FinalBuilder 7, and it expired, you can upgrade by Renewing your subscription.

If you have a license for FinalBuilder 6 or earlier, you can Purchase an upgrade here.

Can I safely install FinalBuilder 8 on a machine with FinalBuilder x?

Yes. FinalBuilder 8 installs into a separate folder (as did all older versions of FinalBuilder).

Can FinalBuilder 8 open an run my old projects?

Yes, FinalBuilder 8 can open and run any projects from any earlier version of FinalBuilder. When you save the projects in FinalBuilder 8, it will save it as a FinalBuilder 8 project (new file).

Can older versions of FinalBuilder open FinalBuilder 8 projects?

No, FinalBuilder 8 projects use a different file format.

Does FinalBuilder 8 work with FinalBuilder Server 7?

No. FinalBuilder 8 does not work with FB Server 7, nor can it be made to work. FinalBuilder Server was discontinued over 2 years ago, if you need Continuous Integration then consider migrating to Continua CI.

I have questions!

Feel free to contact support@finalbuilder.com with any questions you might have.

FinalBuilder 8 Beta

Tue, 05 May 2015 20:06:36 GMT

It's almost 5 years since FinalBuilder 7 was released. Since it's release we have shipped 44 official updates, nearly every update including new features or improvements. This program of continuous improvement has worked well, with customers not having to wait for major new versions to arrive to get support for new versions of Visual Studio or Delphi etc, but it has limited our ability to make major changes. So it's time for a new major version of FinalBuilder.

What's new in FinalBuilder 8

IDE Themes

The IDE has two new themes, Dark and Light (yes, imaginatively named!). The IDE defaults to Dark on first run, however you can change the theme in the options quite easily. The themes are still a work in progress, we are waiting on an update from a third party control vendor to resolve some issues.

Debugger

Actionlists renamed to Targets

Run Target Action

You can now return values from Targets (ie out parameters) .

New Help System

Non Visible Changes

Stepping Engine

Variables Architecture

Core Messaging

Changes to the messaging has allowed us to improve the performance of the stepping engine and logging, with much less thread switching. This also improved the IDE performance.

CLR Hosting

The minimum CLR version is now .NET 4.0 (ie FinalBuilder requires .net 4.0 to be installed).

Code Changes

FBCmd

The command line parameters have changed to be more consistent and easier to specify. You can also specify one or more targets to execute (when not specified, the default target is executed).

New Project File Formats

FinalBuilder has used an xml file format since version 1, however a common complaint over the years, has been that it is difficult to diff file versions. FinalBuilder 8 has tackled this in two ways.

A new DSL style project file format (.fbp8) is now the default format, it is very easy to diff.

project
begin
    projectid = {04710B72-066E-46E7-84C7-C04A0D8BFE18}
    target
    begin
        name = Default
        targetid = {E6DE94D6-5484-45E9-965A-DB69885AA5E2}
        rootaction
        begin
            action.group
            begin
                id = {D860420B-DE46-4806-959F-8A92A0C86429}
            end
        end
    end
end

A new xml format (.fbx8), much less verbose than the old format.



    
        {6A717C24-D00F-4983-9FD0-148B2C609634}
        
            Default
            {E6DE94D6-5484-45E9-965A-DB69885AA5E2}
            
                
                    {D860420B-DE46-4806-959F-8A92A0C86429}

Compressed project files (.fbz8) use the dsl format internally (compressed projects are just a zip file with a project.fbp8 inside it).

The default project file encoding is now UTF-8, which is more version control friendly (some version control systems treat utf-16 as binaries).

New Actions

There are no new actions at the moment, although several are in development, they will be added to the beta builds as they are completed.

How do I get the Beta?

Links to the beta downloads will be published to the FinalBuilder Downloads page.

What if I find a bug?

We have created a Beta forum on our forums, or you can email support (please added Beta to the subject). When reporting an issue, be sure to include the beta build number and details about your environment. Please test with the latest beta build before reporting bugs.

We are particularly keen for people to load up their existing projects from older (ie 7 or earlier) versions of FinalBuilder, save them in FB8 format, and load them again and confirm that everything loaded ok.

When will it be released?

When it's ready ;)

Introducing VSoft.CommandLineParser for Delphi

Sat, 26 Jul 2014 14:20:12 GMT

Command line parsing

Pretty much every delphi console application I have ever written or worked on had command line options, and every one of the projects tried different ways for defining and parsing the supplied options. Whilst working on DUnitX recently, I needed to add some command line options, and wanted to find a nice way to add them and make it easy to add more in the future. The result is VSoft.CommandLineParser (copies of which are included with the latest DUnitX).

Defining Options

One of the things I really wanted, was to have the parsing totally decoupled from definition and the storage of the options values. Options are defined by registering them with the TOptionsRegistry, via TOptionsRegistry.RegisterOption<T> - whilst it makes use of generics, only certain types can be used, the types are checked at runtime, as generic constraints are not flexible enough to specify which types we allow at compile time. Valid types are string, integer, boolean, enums & sets and floating point numbers.

Calling RegisterOption will return a definition object which implements IOptionDefinition. This definition object allows you to set various settings (such as Required). When registering the option, you specify the long option name, the short option name, help text (will be used when showing the usage) and a TProc<T> anonymous method that will take the parsed value as a parameter.

procedure ConfigureOptions;
var
  option : IOptionDefintion;
begin
  option := TOptionsRegistry.RegisterOption<string>('inputfile','i','The file to be processed',
    procedure(value : string)
    begin
        TSampleOptions.InputFile := value;
    end);
  option.Required := true;

  option := TOptionsRegistry.RegisterOption<string>('outputfile','o','The processed output file',
    procedure(value : string)
    begin
        TSampleOptions.OutputFile := value;
    end);
  option.Required := true;

  option := TOptionsRegistry.RegisterOption<boolean>('mangle','m','Mangle the file!',
    procedure(value : boolean)
    begin
        TSampleOptions.MangleFile := value;
    end);
  option.HasValue := False;

  option := TOptionsRegistry.RegisterOption<boolean>('options','','Options file',nil);
  option.IsOptionFile := true;
end;

For options that are boolean in nature, ie they have do not value part, the value passed to the anonymous method will be true if the option was specified, otherwise the anonymous method will not be called. The 'mangle' option in the above example shows this scenario.

You can also specify that an option is a File, by setting the IsOptionFile property on the option definition. This tells the parser the value will be a file, which contains other options to be parsed (in the same format as the command line). This is useful for working around windows command line length limitations.

Currently the parser will accept
-option:value
--option:value
/option:value

Note the : delimiter between the option and the value.

Unnamed parameters are registered via the TOptionsRegistry.RegisterUnNamedOption<T> method. Unlike named options, unnamed options are positional, but only when more than one is registered, as they will be passed to the anonymous methods in the order they are registered.

Parsing the options.

Parsing the options is as simple as calling TOptionsRegistry.Parse, which returns a ICommandLineParseResult object. Check the HasErrors property to see if the options were valid, the ErrorText property has the parser error messages.

Printing Usage

If the parser reports errors, then typically you would show the user what the valid options are and exit the application, e.g:

    parseresult := TOptionsRegistry.Parse;
    if parseresult.HasErrors then
    begin
      Writeln(parseresult.ErrorText);
      Writeln('Usage :');
      TOptionsRegistry.PrintUsage(
        procedure(value : string)
        begin
          Writeln(value);
        end);
    end
    else
        ..start normal execution here

The TOptionsRegistry.PrintUsage makes it easy to print the usage to the command line.

When I started working on this library, I found some really complex libraries (mostly .net) out there with a lot of options, but I decided to keep mine as simple as possible and only cover off the scenarios I need right now. So it's entirely possible this doesn't do everything people might need, but it's pretty easy to extend. The VSoft.CommandLineParser library (just three units) is open source and available on Github, with a sample application and unit tests (DUnitX) included.