VSoft Technologies Blogs

rss

VSoft Technologies Blogs - posts about our products and software development.


Delphi Language Enhancements

TL;DR - The Delphi language is very verbose, dated and unattractive to younger developers. Suggestions for improvements below.

The Delphi/Object Pascal language really hasn't changed all that much in the last 20 years. Yes there have been some changes, but they were mostly just tinkering around the edges. Probably the biggest change was the addition of Generics and Anonymous methods. Those two language features alone enabled a raft of libraries that were simply not possible before, for example auto mocking (Delphi Mocks, DSharp), dependency injection and advanced collections (Spring4D).

Some of the features I list below have the potential to spur on the development of other new libraries which can only be a good thing. I have several abandoned projects on my hard drive that were only abandoned because what I wanted to do required language features that just didn't exist in Delphi, or in some cases, the generics implementation fell short of what was needed. Many of these potential features would help reducing verbosity, which helps with maintainability. These days I prefer to write less lines of more expressive code.

I have tried to focus on language enhancements that would have zero impact on existing code, i.e. they are completely optional. I have often seen comments about language features where people don't want the c#/java/whatever features polluting their pure pascal code. My answer to that is, if you don't like it don't use it!! There are many features in Delphi that I don't like, I just don't use them, but that doesn't mean they should be removed. Each to his own and all that. Another argument put forward is "feature x is just syntactic sugar, we don't need it". That's true, in fact we don't need any language if we are ok with writing binary code! If a feature is sugar, and it helps me write less code, and it's easier to read/comprehend/maintain, then I love sugar, load me up with sugar.

Inspiration

Lots of the examples below borrow syntax from other languages, rather than try to invent some contrived "pascalish" syntax. The reality is that most developers these days don't just work with one programming language, they switch between multiple (I use Delphi, C#, Javascript & Typescript on a daily basis, with others thrown in on occasion as needed). Trying to invent a syntax just to be different is pointless, just borrow ideas as needed from other languages (just like they did from delphi in the past!) and get on with it!

I have not listed any functional programing features here, I have yet to spend any real time with functional programming, so won't event attempt to offer suggestions.

I don't have any suggestions for how any of these features would be implemented in the compiler, having not written a real compiler before I know my limitations!

Ok, so lets get to it, these are not listed in any particular order.

Local Variable Initialisation

Allow initialisation of local variables when declaring them, eg :

	procedure Something;
	var
	  x : integer = 99;
	begin
	.........
	

Benefits : 1 less line of code to write/maintain, per variable, initial value is shown next to the declaration, easier to read.

Type Inference

	var
	  x = 99; //it's an integer
	  s = 'hello world'; //it's a string
	begin
	........
	

Benefits : less ceremony when declaring variables, still easy to understand.

Inline variable declaration, with type inference and block scope

	procedure Blah;
	begin
      .......
	  var x : TStrings := TStringList.Create; //no type inference
	//or
	  var x := TStringList.Create; // it's a TStringList, no need to declare the type
	  .......
	end;
	

Inline declared variables should have block scope :

	if test = 0 then
	begin
	  var x := TListString.Create;
	  ....
	end;
	x.Add('bzzzzz'); //Compiler error, x not known here!
	

Benefits : Declare variables when they are needed, makes it easier to read/maintain as it results in less eye movement, block scope reduces unintended variable reuse.

Loop variable inline declaration

Declare your loop or iteration variable inline ( and they would have loop block scope)

	for var item in collection do
	begin
	  item.Foo;
	end;
	item.Bar; //<<error item unknown.
	
	for var i : integer := 0 to count do
    ....
	//or
	for var i := 0 to count do //using type inference
    ....
	

Benefits : Avoid the old variable loop value not available outside the loop error, same benefits as inline/block scope etc.

Shortcut property declaration

Creating properties that don't have getter/setter methods in Delphi is unnecessarily verbose in Delph

	type
	  TMyClass = class
	  private
	    FName : string;
	  public
	    property Name : string read FName write FName;
	  end;
	

All that is really needed is :

	type
	  TMyClass = class
	  public
	    property Name : string;
	  end;
	

Whilst this might seem the same as declaring a public variable, RTTI would be generated differently if it was a variable rather than a property.

Benefits : Cuts down on boilerplate code.

Interface Helpers

Add interface helpers just like for classes and records. Also, remove the limit of one helper per type per unit (without this the above is not really usable). Have a look at how prevalent extension methods are in C#, Linq is a prime example, it's essentially just a bunch of extension methods.

	type
	  TIDatabaseConnectionHelper = interface helper for IDatabaseConnection
	    function Query<T> : IQuerable;
	  end;
	

The above is actually a class that extends the interface when it's containing unit is used. This would make implementing something like LINQ possible.

Strings (and other non ordinals) in Case Statements

This has to be the most requested feature by far in the history of delphi imho, sure to make many long time Delphi fans happy.

	case s of
	  'hello' : x := 1;
	  'goodbye' : x := 2;
	  sWorld : x := 3; //sWorld is a string constant
	end;
	

Case sensitivity could be possibly be dealt with at compile time via CaseSensitive 'helper', (or perhaps an attribute). The above example is case insensitive, the example below is case sensitive :

	case s.CaseSensitive of
	  'hello' : x := 1;
	  'goodbye' : x := 2;
	  sWorld : x := 3; //sWorld is a string constant
	end;
    //or
	case [CaseSensitive]s of
	  'hello' : x := 1;
	  'goodbye' : x := 2;
	  sWorld : x := 3; //sWorld is a string constant
	end;
	

How about :

	case x.ClassType of
	  TBar : x := 1;
	  TFoo : x := 2;
	end;
	

Benefits : Simpler, less verbose code.

Ternary Operator

A simpler way of writing :

	If y = 0 then
	  x := 0
	else
	  x := 99;
	

Syntax : x := bool expr ? true value, false value

	//eg
	x := y = 0 ? 0 : 99;
	

Benefits : Simpler, more succinct code.

Try/Except/Finally

Probably the equal most requested language feature, allowing for try/except/finally without nesting try blocks.

	try
	...
	except
	..
	finally
	...
	end;
	

Much cleaner than:

	try
	  try
	    ...
	  except
	    ...
	  end;
	finally
	  ....
	end;
	

Benefits : Neater, Tidier, Nicer

Named Arguments

Say for example, we have this procedure, with more than one optional parameter :

	procedure TMyClass.DoDomething(const param1 : string; const param2 : ISomething = nil; const param3 : integer = 0);
	

To call this method, if I want to pass a value for param3, I have to also pass a value for parm2

	x.DoSomething('p1', nil,99);
	

With named parameters, this could be :

	x.DoSomething('p1', param3 = 99);
	

Yes, this means I have to type more, but it's much more readable down the track when maintaining the code. I also don't need to lookup the order of the parameters, or provide redundant values for parameters that I want to just use their default values for. I also don't end up adding a bunch of overloaded methods to make the method easier to call.

Benefits : More expressive method calls, less overloads.

Variable method arguments

Steal this from C# (which probably borrowed the idea from c++ varargs ... feature)

	procedure DoSomething(params x : array of integer);
	

The method can be called passing ether an array param

	procedure TestDoSomething;
	var
	   p : array of integer;
	begin
	...... //(fill out the p array)
	  DoSomething(p);
	  //or
	  DoSomething(1);
	  DoSomething(1,2);
	  DoSomething(1,2,3);
	  ........
	

Benefits : Flexibilty in how a method is called.

Lambdas

Delphi's anonymous method syntax is just too damned verbose, wouldn't you prefer to write/read this:

	var rate := rates.SingleOrDefault(x => x.Currency = currency.Code);
	

rather than this :

	var
	  rate : IRate;
	begin
	  rate := rates.SingleOrDefault(function(x : IRate) : boolean<
	                                begin
	                                  result := x.Currency = currency.Code;
	                                end);
	...
	

Ok, so I did sneak in an inline type inferenced local var in the first example, and both snippets rely on the existence of interface helpers (rates would be IEnumerable>Rate<) ;)

Benefits : Less code to write/maintain, the smiles on the faces of developers who switch between delphi and c# all the time!

LINQ!

With Lambdas and multiple type helpers per type per unit, a LINQ like feature would possible. Whilst the Delphi Spring library has had Linq like extensions to IEnumerable<T> for a while, this would formalise the interfaces and provides an implementation used by the collection classes. Providers for XML and Database (eg FireDac) would be possible.

Even a Linq 2 VCL & Linq 2 FMX would be possible. A common scenario, update a bunch of controls based on some state change :

	Self.Controls.Where( x => x.HasPropertyWithValue<boolean>('ForceUpdate', true) or x.IsType<IForceUpdate>).Do( c => c.Refresh);  //or something like that.
	

This would make it possible to operate on controls on a form, without a) knowing their names, b) having references to them, or c) knowing if the control actually exists.

Being able to do this with something like a linq expression would be a massive improvement.. all that's needed is a linq provider for each control framework. Sorta Like jQuery for the VCL/FMX!

Benefits : Too many to list here!

Caveats : Microsoft have a patent on LINQ - so perhaps this isn't really doable? Perhaps with a slightly different syntax. Or challenge the patent!

Async/Await

Bake the parallel library features into the language/compilers, much like C# and other languages have done over recent years (ala async, await). That makes it possible/easier to write highly scalable servers, over the top of asynchronous I/O. Yes, it's technically possible now, but it's so damned difficult to get right/prove it's right that very few have attempted it. This also make it possible to use the Promise pattern, something along these lines - https://github.com/Real-Serious-Games/C-Sharp-Promise to handle async tasks.

Benfits : Write scalable task oriented server code without messing with low level threading, write responsive non blocking client code.

Caveats : Microsoft have a patent on Async/Await

Non reference counted interfaces

Make it possible to use have interfaces without reference counting. I use interfaces a lot, even on forms and frames, I like to limit what surface is exposed when passing these objects around, but it can be painful when the compiler tries to call _Release on a form that is being destroyed. Obviously, there are ways to deal with this (careful clean up) but it's an easy trap to fall in and very difficult to debug.

Possible syntax :

	[NoRefCount] // << decorate interface with attribute to tell compiler reference counting not required.
	IMyInterface = interface
	....
	end;
	

There would have to be some limits imposed (using the next feature in this list), for example not allowing use of the attribute on an interface that descends from another interface which is reference counted. It would cause mayhem if passed to a method that takes the base interface (for which the compiler would then try generate addref/release calls).

Benefits : Remove the overhead of reference counting, avoid unwanted side effects from compiler generated _Release calls.

Attribute Constraints

(RSP-13322)

Delphi attributes do not currently have any constraint feature that allows the developer to limit their use, so for example an attribute designed to be applied to a class can currently be applied to a method.

Operator overloading on classes.

Currently operator overloading is only available on records, adding them to classes would make it possible to create some interesting libraries (DSL's even).

I had discussions with Allen Bauer (and others over) many years ago about about this. Memory management was always the stumbling block, with ARC this would not be big issues. Even without ARC, I think delphi people are capable of dealing with memory management requirements, just like we always have.

Edit : I believe this feature is actually available on the ARC enabled compilers.

Improve Generic Constraints

Add additional constraint types to generics, for example :

	type
	  TToken<T:enum> = class
	    .....
	  end;
	

Benefits : more type safety, less runtime checking code required.

Fix IEnumerable<T>

Not so much a language change rather than a library change, please borrow spring4d's version - much easier to implement in 1 class (delphi's version is impossible to implement in 1 class as it confuses the compiler!).

Yield return - Iterator blocks

The yield keyword in C# basically generates Enumerator implementations at compile time, which allows the developer to avoid writing a bunch of enumerator classes, which are essentially simple state machines. These enumerators should be relatively easy for the compiler to generate.

This a a good example (sourced from this stackoverflow post

	public IEnumerable<T> Read<T>(string sql, Func<IDataReader, T> make, params object[] parms)
	{
	  using (var connection = CreateConnection())
	  {
	    using (var command = CreateCommand(CommandType.Text, sql, connection, parms))
	    {
	      command.CommandTimeout = dataBaseSettings.ReadCommandTimeout;
	      using (var reader = command.ExecuteReader())
	      {
	        while (reader.Read())
	        {
	          yield return make(reader);
	        }
	      }
	    }
      }
	}
	

The example above will read the records in from the database as they are consumed, rather than reading them all into a collection first, and then returning the collection. This is far more memory efficient when there are a lot of rows returns. In essence, the consumer/caller is "pulling" the records from the database when required, this could for example be pulling messages from a queue.

I guess for delphi, this could be a simple Yield() method( or YieldExit() to more more explicit).

Benefits : less boilerplate enumerator code, lower memory usage when working with large or unbounded datasets or queues etc.

Partial classes

Yes I know, this one is sure to kick up a storm of complaints (I recall an epic thread about this topic on the newsgroups about this years ago). Like everything else, if you don't like it, don't use it. I don't often use this feature in C#, but it is indispensable for one particular scenario, working with generated code. Imagine generating code from an external model or tool for example type libraries, uml tools, database schema, orm, idl etc. The generated code would typically be full of warning comments about how it's generated code and shouldn't be modified. Partial classes get around this, by allowing you to Add code to the generated classes, and the next time they are regenerated, your added code remains intact. Simple, effective.

Benefits : Enables code generation scenarios without requiring base classes.

Allow Multiple Uses clauses

(RSP-13777)

This is something that would make commenting/uncommenting units from the uses clause a lot easier, and also make tooling of the uses clause easier.

	uses
	sharemem, System.SysUtils, System.Classes, vcl.graphics{$ifdef debug},logger{$endif};
	

The above syntax is easily messed up by the IDE.

	uses sharemem;
	uses System.SysUtils, System.Classes;
	uses vcl.graphics;
    {$IFDEF debug}uses logger;{$endif}
	

This makes commenting-out and reorganising the library names more convenient. It would also make refactoring and other tooling easier.

Allow non parameterized interfaces to have parameterized methods

(RSP-13725)

	IContainer = interface
	['{2B7B3956-7101-4619-A6DA-C8AF61EE4A81}']
	  function Resolve<T>: T;
	end;
	

That won't compile, but this code works as expected:

	TContainer = class
	  function Resolve<T>: T;
	end;
    

This is a limitation I have come across many times when trying to port code from C# to Delphi.

Conclusion

That's 20+ useful, optional and simple to implement (just kidding) language features that I would personally like to see in Delphi (and I would use every one of them). I could have gone on adding a lot more features, but I think these are enough to make my point.

Here's my suggestion to Embarcadero, invest in the language and make up for lost time! While you are at it, sponsor some developers to try porting some interesting and complex open source libraries to delphi, and when they hit road blocks in the language or compiler(and they will), make it work, iterate until baked. I tried porting ReactiveX http://reactivex.io/ to delphi a while back, but hit too many roadblocks in the language and compiler (internal compiler errors, limitations in the generics implementation).

The end result would be a modern, capable language that can handle anything thrown at it.


Visual Studio 2017 RC Support

Today we released updates to FinalBuilder (8.0.0.2007) and Continua CI (1.8.1.185) which add support for Visual Studio 2017 RC. 

There were a few issues we found with Visual Studio 2017 Release Candidate. The first was, unlike every other previous version of Visual Studio, 2017 RC does not write any useful information to the registry that allows us to reliably find out where it was installed to. So at the moment, we just check the default install locations :

On 64 bit windows :

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE

on 32 bit windows (do people still use it?) :

C:\Program Files\Microsoft Visual Studio\2017\Community\Common7\IDE\Common7\IDE
C:\Program Files\Microsoft Visual Studio\2017\Professional\Common7\IDE
C:\Program Files\Microsoft Visual Studio\2017\Enterprise\Common7\IDE

Yes, each edition now installs into different folders. 

As a work around for those who don't install in the default location, you can set environment variables  MSBuild15Path (for MSBuild) and  VisualStudio2017Path for Visual Studio, if we don't find VS/MSBuild in the default locations, we'll check for those environment variables.  

Another issue we found is that when building with devenv.com (ie the visual studio command line, rather than msbuild), builds take a looooong time.. it appears to hang after building, and it also seems to randomly fail with windows error 80131623 (FatalExecutionEngineError). This has nothing to do with FinalBuilder or Continua CI, it happens on the command line as well. I have reported these issues to microsoft, hopefully they will resolve them before release. On the MSBuild Action in FinalBuilder, we were not able to get the list of targets for a dotnet core project. This is still being investigated, we do still hope to resolve this eventually. 

Note, since this work was done against a Release Candidate, things may change between now and the VS2017 RTM release (microsoft seem to have redifined what a release candidate is lately!). If you find any issues let us know.

BTW, TFS 2017 support is currently being worked on for both FinalBuilder and Continua CI. We should have a VSO FinalBuilder build task available in the marketplace soon (it's close). However, getting our xaml builds working with TFS2017 is proving challenging (just like previous releases), there is no doco and it's deprecated (good riddance) and visual studio won't load our projects.. stay tuned.




This version adds several back-end performance enhancements including significant improvements to database query caching. We have also updated various third party packages and upgraded the bundled PostgreSQl database from version 9.3.4 to version 9.5.3. What’s more, this release builds upon all the improvements and fixes made to version 1.8.

New Actions


To keep up with the latest developments in Visual Studio, we have added several new actions for working with .Net Core projects. The DotNet Build, DotNet Pack, DotNet Publish, DotNet Restore, DotNet Run and DotNet Test actions all run the DotNet Cli command line.

DotNet Build Action

This release also includes a new action for importing XUnit tests from an XML report file. The sort of file you may choose to output from the DotNet Test action.

To polish this off we have added new NPM Pack and NPM Publish actions for sharing your completed packages.

New Build Event Handlers


A powerful new addition to the Continua CI build event handlers is the HTTP Request handler which allows you to send JSON or XML to HTTP endpoints via GET, POST. PUT, DELETE, PATCH methods and extract values from the results to set as Continua CI variables.

HTTP Request Build Event Handler

This enables interaction with a various existing web services and REST APIs. You could, for example, use this feature to send a message via Slack when a build starts, tag a commit on GitLab when a build finishes or even translate a variable value to Azerbaijani using Google Translate API. If you have any in-house web services which handle HTTP requests, it is likely that these can be accessed too.

We have also added a new GitHub Release build event handler for creating, updating and deleting GitHub releases. This also allows you to upload artifacts as GitHub release assets.

Other New Features


Build Event Handler conditions. You can now control the triggering of handlers based on the value of build variables or expression objects. These values may be specified when queuing a manual build or set in the stage workflow. You could, for example, only send a GitHub Release if the number of failed unit tests $Build.Metrics.UnitTests.Failed$ is zero.

Build Event Handler conditions

Continue on failure. Builds can now be set to progress to the next stage after one stage has failed. Stage gates now include an $Stage.IsSuccessful$ condition by default. This can be removed to allow the build to continue to the next stage when a stage fails. So you can see what failed, the stages are now coloured green or red in Tile and Details dashboard views to indicate an success or failure status.

Continue on failure

New cleanup options. We have also made some changes to the server cleanup policy, giving you the options to clean up build statistics. The database category has also been split up so that build unit tests can be cleaned up without cleaning up the full build. This is important to prevent database tables growing too large when you have a large number of unit tests.

Cleanup options


Continua CI 1.8.0.176 and FinalBuilder 8.0.0.1817 (both released today) provide somewhat better integration between the two products.  

Continua CI 1.8

The FinalBuilder Action in Continua CI now produces an xml file that is consumed by FinalBuilder when run under Continua CI. This xml file contains all the useful information about a build, such as version numbers, changesets, variables etc. 

There is also a new option on the action that will automatically apply Continua CI variable values to matching FinalBuilder variables. What this means is, if you have a variable declared in both FinalBuilder and Continua CI with the same name, FinalBuilder's variable will automatically get the value of the Continua CI variable at runtime. This option is only available for FinalBuilder 8, if you select FinalBuilder 7 the option will not be visible. If the version of FinalBuilder 8 you are running does not support this integration, a warning will appear in the Continua CI build log. 

FinalBuilder 8

FinalBuilder 8 has two new actions.

The "Is Running Under Continua" action is an If Then style action, i.e. the children of this action will only run if FinalBuilder is running under Continua CI.  


The other action is the Continua CI - Get Version Info action - this action will take the version info from Continua CI and apply it to a version info property set in your FinalBuilder project.;

 

The action is smart enough to use the correct version info scheme depending on whether the propertyset is a win32 or dotnet propertyset. This greatly simplifies getting the version info from Continua into FinalBuilder, no need to declare 4 variables on both sides and set them in the FinalBuilder Action in Continua CI.

Scripting

That xml file I mentioned above is loaded into a Continua object model that is available in action script events. If you do make use of it, you should be sure to check the Continua.IsRunningUnderContinua property before using the rest of the object model (the script editor provides intellisense for the model.

We have been working on moving Continua CI to .net 4.6.1 for a future release, and during this conversion (so far, mostly just updating nuget packages),  we discovered an issue that turned out to be caused by a change to .net certificate validation.

If you use self signed X509 certificates and target the .net framework 4.6.1 then you are in some fun, especially if you used makecert to generate the certificate. There is a change in behaviour in the way certificates are validated, which will leave you pulling you hair out for hours. The error you may encounter will look something like this : 

"The Identity check failed for the outgoing message. The remote endpoint did not provide a domain name system (DNS) claim and therefore did not satisfied DNS identity 'localhost'. This may be caused by lack of DNS or CN name in the remote endpoint X.509 certificate's distinguished name."

If you google the error message, you will find plenty of references to using  a DnsEndpointIdentity, only in our case, we were already doing exactly as the answers on stackoverflow were suggesting! Since we were migrating from .net 4.0 to .net 4.6.1, I started looking for info on the changes in each version of the .net framework. Eventually, I came across this page :

https://msdn.microsoft.com/en-us/library/mt620030(v=vs.110).aspx

This was the only mention of X509 certificates I could find in the change history, but it seems like it could be related, so I tried what it suggested, and low and behold, problem solved! With some further investigation of this work around, I found some issueson the wcf github repo with several references to the behaviour of certificate validation.

https://github.com/dotnet/wcf/issues/321

https://github.com/dotnet/wcf/pull/603

So it turns out, in .NET 4.6.1 they broke the certificate validation, by only looking at the Subject Alternate Name (SAN) extension, and not falling back to the Subject Name CN field as it should. The pull request I linked to above, is for the WCF for .NET core, not 4.6.1 - so I have no way of knowing if this will be fixed for 4.6.x.  

Whilst an app.config change can work around the issue, the real fix is to generate certificates that include SAN extension.

Generating a certificate without MakeCert

Makecert.exe, which is commonly used (on windows at least) does not support the SAN extension, Makecert is deprecated, so it's unlikely there will be an update to make it able to generate certificates with the SAN certificate extension.  Windows 8/Server 2012R2 & later versions of Windows include a new powershell cmdlet to generate certificates. For Windows 7 the best option is this :

https://gallery.technet.microsoft.com/scriptcenter/Self-signed-certificate-5920a7c6

This appears to what the Windows 8 & later cmdlet is based on, it's relatively simple to use and generates certificates that validate properly with WCF on .NET 4.6.1

New-SelfSignedCertificateEx -Subject "CN=MyServer" -KeySpec Exchange 
-KeyUsage "DataEncipherment, KeyEncipherment, DigitalSignature" 
-Path c:\certs\example.pfx -Exportable -SAN "MyServer" -SignatureAlgorithm sha256 
-AllowSMIME -Password (ConvertTo-SecureString "abc123dontuseme" -AsPlainText -Force) 
-NotAfter (get-date).AddYears(20)

Note the above command is on multiple lines to make it easier to read, it can be on one line.