Hi,
we currently use 4 core desktop CPUs in our Build servers.
In order to use these our FB scripts define a queue with all projects that need to be built and then use an async action group to start 4 tasks that each contain a queue iterator.
(I have attached a sample project to better illustrate what I mean)
This works on our old build servers but now management is planning to buy a couple new 24 core machines.
I could manually extend the project to use 24 tasks but this would mean a lot of work and would not be particularly future proof either.
So I was wondering if anyone had any experience with using many cores in FB and could share it with me.
Regards,
Christoph__42138__0__ParallelBuild.zip (1.679 KB)
Hi Christoph
Adding more threads would not necessarily improve performance, or get more work done in the same time period. Most builds are I/O bound, so at some point you will hit a roadblock where more thread don’t improve things, and may even make things worse. Running multiple threads is not free, there is a cost associated with thread/context switching, and logging in particular becomes a bottleneck.
Your project looks to be based on this example - https://github.com/VSoftTechnologies/FinalBuilder.Examples/tree/master/FB8/ParallelQueue
On my 8 core desktop machine, I get pretty much the same result with 2 or 4 or 8 threads, but when that example is actually put to work running real tasks, which use i/o and cpu, the results are extremely variable.
The only real advice I can give on this would be to try it.
FWIW, we’ve been load testing/profiling/optimising Continua CI for the last few weeks. We have made some significant improvements to cpu usage and database i/o, but it’s a lot hard to improve file system i/o performance, that eventually comes down to hardware choices. If you haven’t read this blog post (mostly about Continua, but some of it applies to FinalBuilder as well.
TL;DR - Buy Fast SSD’s, that will give you more bang for buck than buying more cpu cores!
I understand that fast I/O quickly becomes more important than more threads but sadly hardware selection is beyond my control.
Your answer confirmed my suspicions though.
I’ll see if I can’t at least get a few SSDs added to the machines so we can run multiple build VMs with fewer cores simultaneously instead.