Signing New .rdp says the Format of the subject is not recognised

I’ve been using Signotaur to sign both .EXEs and .RDP files successfully. Yesterday, I created a new .rdp file and tried to sign it but got a new error with the following reason: Unknown trust subject form. The format of the subject is not recognised (TRUST_E_SUBJECT_FORM_UNKNOWN).

I’ve looked through the knowledgebase and troubleshooting tips but am not sure what to do.

Hi David,

That error (TRUST_E_SUBJECT_FORM_UNKNOWN ) comes from Windows and indicates the file was passed to the standard Authenticode signer (used for .exe /.dll /.msi files), which doesn’t recognise the format.

Signotaur selects the RDP signing path purely from the file extension, so could you confirm the file is actually named .rdp (for example, Windows Explorer may be hiding the real extension)?

Also, please confirm you’re running the latest version of SignotaurTool by running signotaurtool --version.

If the file genuinely has a .rdp extension and you’re on a current build, please send through the full command line along with a verbose client log (-v ) and we’ll investigate further. You can send the log privately via direct message or email it to support@finalbuilder.com.

I have Signotaur version 1.1.0.69 but “–version” isn’t recognized:

+----------------------------------------------------------------------------------------------------------------------+
|                                                                                                                      |
|  SignotaurTool                                                                                                       |
|  Version: 1.1.0.69 (Source: 3e5b52661404badab356774574ab40f00404cf2021cfb6dee6b814892fff635d)                        |
|  © 2026 VSoft Technologies Pty Ltd                                                                                   |
|                                                                                                                      |
+----------------------------------------------------------------------------------------------------------------------+

Error: Unexpected option 'version'.

       --version
       ^^^^^^^^^ Did you forget the command?

Yes, the file has .rdp extension. It also has spaces in it but I put quotes around it. I even renamed the file with underscores instead of spaces and left off the quotes and still got the same error.

I use a batch file to do all my signing so I use the same settings all the time. It has the API key and thumbprint embedded in temporary environment variables; therefore, the part of the batch file that calls Signotaur is:

"c:\utils\SignotaurTool.exe" sign --nb --api-key %SignAPI% --thumbprint %SignThumb% --sign-server https://goliath:8087 --file-digest SHA256 --tr http://ts.ssl.com --td SHA256 --verify-cert-chain --allow-untrusted %1

(my computer is named “goliath”)

I’ll send the log shortly.

Hi David,

You’re right - apologies, there isn’t a --version argument. The tool prints the version automatically when run without a command.

From your output, you’re running SignotaurTool v1.1.0.69, which is quite an old build. RDP signing support was added later in v1.2.0.107

So the behaviour you’re seeing is expected on that version - .rdp files are being passed through the normal Authenticode signing path, which results in the TRUST_E_SUBJECT_FORM_UNKNOWN error.

You’ll need to upgrade both the server and client to a current build. Once the server is updated, the latest SignotaurTool.exe can be downloaded directly from the server home page, or you can update the existing client using the SignotaurTool.exe update command.

While updating the batch file, you might also want to add --auto-self-update (or the short form --asu ) to the sign command line. With that enabled, SignotaurTool checks for and installs updates before each signing run.

After I sent the log, I was checking the version history (should’ve done that first) and realized, as you pointed out, I have an old version. I just updated server and client but something is still not right:

16:39:52 Debug: Built RDP canonical content (1176 bytes, 11 signed fields)
16:39:52 Error: Unexpected error while RDP signing C:\utils\ABS_Cloud_Gateway.rdp
System.ArgumentException: Certificate does not have an RSA public key.
   at VSoft.Signotaur.Client.Signing.RemoteSigningKey..ctor(SignService.SignServiceClient client, X509Certificate2 certificate, String apiKey, String thumbprint, String fileName, String fileLastModifiedUtc, Int64 fileSizeInBytes, String fileVersion, ILogger logger, Boolean showTimings) + 0x10c
   at VSoft.Signotaur.Client.Signing.Rdp.RdpSignatureBuilder.BuildSignature(Byte[] canonicalBytes, SigningContext context, String fileName, Int64 fileSizeInBytes, String fileLastModifiedUtc, ILogger logger) + 0xa7
   at VSoft.Signotaur.Client.Signing.Rdp.RdpSigner.<SignRdpFileAsync>d__6.MoveNext() + 0x282
16:39:52 Debug: RDP signing operations were cancelled.
16:39:52 Debug: Returning exit code: 4.
16:39:52 Error: Exit code: 4

As a quick check, I just signed an executable with this batch file I use, now using the most up-to-date version and it signed successfully.

Hi David,

That’s a different issue - the new RDP signing path requires an RSA certificate, and GetRSAPublicKey() is returning null for the certificate you’re using. In practice, that almost certainly means the certificate is ECDSA (less likely DSA). The error message isn’t very helpful (we’ve just improved it for the next release so it’ll name the actual key type, e.g. ECDSA, instead of just saying “not RSA”).

Currently the RDP, CMS, NuGet, VSIX and manifest signers all require RSA keys. Only the Authenticode path currently supports non-RSA keys. This limitation comes from Microsoft’s signing platform support for those formats.

You can confirm the key type in the Signotaur admin UI under Certificates - the Key Type column for the thumbprint you’re using will show whether the certificate is RSA or ECDSA.

If it is ECDSA, you’ll need to register an RSA code-signing certificate and point %SignThumb% at that certificate’s thumbprint instead.

Thanks for the explanation; I don’t fully understand all this but I’ll do some research.

The really strange thing is that a month ago, I code-signed a different .rdp file with the older version of Signotaur that I used earlier today and that same batch file–and it worked! There were two lines added to the end of the .rdp file: signscope and signature.

From what you’re telling me (and I read in the version history), that shouldn’t have been possible! Yet, I have the file and when I used it to connect to a server, the Publisher was no longer “unknown” but showed my company name, signifying that it was truly code-signed. Now I’m really puzzled!

Hi David,

Honestly, it shouldn’t have been possible. SignotaurTool v1.1.0.69 had no RDP signing path at all, which is why your earlier attempt produced the TRUST_E_SUBJECT_FORM_UNKNOWN error.

So whatever produced the working .rdp file from a month ago was almost certainly not SignotaurTool v1.1.0.69. The most likely explanation is that it was signed with rdpsign.exe (Microsoft’s own RDP signing tool in C:\Windows\System32 ), either manually, from another script, or perhaps on another machine.

The easiest way to confirm is to run:rdpsign.exe /v "<working file>.rdp" from an elevated command prompt. That will print the thumbprint of the signing certificate used. If it matches an RSA certificate you already have access to, that’s likely the certificate you’ll need to register with Signotaur for RDP signing.

:man_facepalming: Wow–I must be losing my mind! You’re exactly right–I used rdpsign. I think I did try Signotaur but got an error (as expected). At that point, I didn’t even know that signing a .rdp file was possible so was just reading and trying things. I think I used the thumbprint I used for signing with signotaur as the /sha256 parameter to rdpsign. I have no idea if that’s how it should be done but it seemed to work.

FWIW, it just used rdpsign with this newest .rdp and it added those two lines to the end again, except that the signature was significantly longer for this file.

Anyway, thanks for your patience and explaining things to me.

Ha - no worries, glad we got to the bottom of it.

That does raise an interesting question though: if rdpsign worked with that thumbprint, then the certificate in your local Windows store must be RSA (rdpsign would have failed otherwise).

So either a different certificate was registered with Signotaur than the one you used with rdpsign, or perhaps a different thumbprint ended up being used than expected.

Could you check in the Signotaur admin UI under Certificates? What does the Key Type column say for the row matching your %SignThumb% thumbprint? RSA or ECDSA?

That’ll quickly tell us whether you need to register a different certificate to get RDP signing working with Signotaur, or whether something stranger is going on.

Sorry for the late reply. I forgot to check back for responses and didn’t get an email about a new message. I just happened to see the indicator of a new message and just now saw your question.

Yes, I had the same question in the back of my mind: if rdpsign works but Signotaur doesn’t, what gives?

I have one certificate installed in Signotaur that I’ve used to code-sign executables. The Key Type is ECDSA. Hmm… doesn’t that imply that using that with rdpsign would not work?

Hi David,

We’ve investigated the RSA requirements for .rdp signing further and, while what I said previously is true for SignotaurTool’s current implementation - the existing RDP signing support only accepts RSA certificates, so ECDSA certificates are rejected up front - it turns out the .rdp format itself does not inherently require RSA. The signature is a detached PKCS#7/CMS structure, which is algorithm-agnostic.

We’ve now added ECDSA support for .rdp signing and, in testing, an ECDSA-signed .rdp file opens directly into the remote session with no security warning dialog, same as RSA. That support will be included in an upcoming release.

That said, we’d still generally recommend RSA unless you have a specific reason to use ECDSA. Microsoft’s Trusted Root Program currently lists ECDSA as “Not Supported” for code signing.

So ECDSA will be a supported option for .rdp signing specifically, but RSA remains the safest and most universally-compatible choice overall.

Interesting. When I purchased my code-signing cert from ssl.com, I was just looking for the cheapest way to digitally sign .EXEs I distribute; I didn’t check to see which type it was, just that it would work.

When I go to renew, I’ll do a little more research. And I’ll start with this comparison of ECDSA and RSA.

By the way, RDP still gives me a security warning even though it’s signed. The only difference is that instead of saying “Publisher: Unknown” it now lists my company name. I might need to buy a more expensive certificate next time.

That part is expected. Signing the .rdp file changes the warning from “Publisher: Unknown” to your company name, but to remove the warning dialog entirely the client machine also needs to trust the publisher certificate.

That can be done either by adding the certificate thumbprint to the trust policy or by enabling the relevant Group Policy settings. Details are covered here: RDP file signing

Someone was telling me about that last night. Thanks for the link to that article; those PowerShell scripts will come in handy!