Stage run under WSL

Continua CI Feature Requests
1

Our product is moving towards Linux and since Continua does not currently support Linux, we are force to pass some of the build system through WSL using WSL commands.

Since file system emulation is really slow on WSL using windows workspace to run WSL commands takes ages, so we need instead first copy our workspace to WSL and then run under WSL.

I wonder if it would be possible to run a stage or whole configuration under WSL, so that all actions would actually be ran in WSL environment. So whole workflow repo copy → workspace copy → actions → workspace copy → artifact would work.

2

HI Michal,

Your diagnosis of the performance issue is correct. Repeatedly accessing a Windows workspace through /mnt/c can be very slow.

It is not currently possible to run an entire stage or configuration within WSL. Continua actions are executed by the Windows agent service, and there is no mechanism to hand a stage over to a separate WSL execution environment. Native Linux agents are on our roadmap, although I cannot provide a timescale at this stage.

Pointing the Continua workspace at \\wsl$\... would be unlikely to help. It would make the files faster for WSL processes to access, but operations performed by Continua, such as repository checkout, workspace copying, artifact collection and cleanup, would then have to access the WSL filesystem across the same Windows and WSL boundary. In addition, \\wsl$ is associated with a user’s WSL session and may not be accessible to the account running the Continua agent service.

The approach that does work is to move the files onto WSL’s native filesystem in bulk and run the build there, rather than having the build access every file through /mnt/c. There are two ways to do this.

One option is to run an SSH server within your WSL distribution and treat it as a remote Linux machine:

  1. Use the File Transfer action over SFTP to copy the workspace into WSL.
  2. Use the SSH Run Script action to perform the build.
  3. Use File Transfer again to copy the build output back for artifact collection.

The transfer still crosses between Windows and WSL, but it happens once, and the files are written directly to WSL’s native filesystem.

I would recommend benchmarking this against your current process before committing to it, as the improvement will depend on the size and structure of the workspace.

Alternatively, you can continue invoking WSL directly. Copy the workspace into WSL’s native filesystem once at the start of the stage, then run each subsequent Execute Program action against that copy. The WSL filesystem persists between invocations, so each action can continue working in the same directory. At the end, copy only the required outputs back into the Continua workspace for artifact collection. The overhead of invoking wsl.exe itself is negligible; it is the paths the build operates on that make the difference.

I’ve added your use case to the existing request for native Linux agent support.

1 Like
3

We already do the copy, the speed up is huge, file intensive operations like generating coverage from gcov files can be 10 times faster.

We will stick to scripts then.