This is a general question but I work in the .NET world so I'd like to know if there any specific quirks about the .NET Framework / Core platforms that I should be concerned about here.
I think it's safe to say that as a general rule, breaking long-running jobs in to threads could be slightly faster than breaking jobs in long-running processes. However, the amount of performance difference would be negligible. I haven't done any benchmarks on this, but this answer seems to indicate that there is an consensus on the matter at least as far as .NET is concerned.
However, I still hear the argument "It was broken up in to separate processes for performance reasons" often. I'm puzzled by this and I wonder if this argument ever holds weight. Please remember, this discussion has nothing to do with maintainability. The question I am asking: are multiple jobs running on a single process each necessarily faster than multiple jobs running on a single thread each inside one process?
Intuitively, I would have to guess that the answer is no. For example, 10 long-running jobs running on 10 threads should run with roughly the same performance as 10 long-running jobs on 10 different processes. But, I see people making the design choice of breaking services up in to smaller parts purely for the purpose of performance.
What I suspect is going on is that shoddy code has created scenarios where starvation is occurring. I think that what is happening is that parallelism is being overused and availability of CPUs is not being honoured so that the benefits of multiple cores are being eroded because the jobs are fighting for CPU power. Still, this doesn't really explain why breaking a job out in to separate processes would improve performance.
So, how can I prove that either a) performance is not improved by breaking jobs out in to separate processes, or b) the improvements made by breaking the code out in to separate processes is only because of poor code design in the first place?
Best Answer
There was a time when I believed this as well. But I have first-hand, real-world experience that this can be very wrong - specifically with .NET. I have observed several scenarios where the extra isolation by processes gained a speed-up by a factor almost equivalent to the number of CPUs, whilst thread parallelization only gained a speedup of a factor 2 at maximum. Though I did not always understood the reasons behind this in full, I can tell you two (possible) causes for such behaviour:
Shared memory pool between all threads: not sure how well the garbage collector handles the objects created by different threads, but I am pretty sure this was one of the reasons where the speedup was far below the factor we expected
Usage of third-party libs which support multithreading, but do not support it very efficiently. An example for this is ADODB for accessing MS Access databases. If one uses thread parallelization to open one connection per thread, each one connecting to a different database file, it seems the different threads disturb each other - still working correctly, but way slower than one could expect.
Hence my recommendation is to try it out: if there is a task to solve where you would expect thread parallelization to bring you a speed factor of "N", where N = "no of CPUs", but you only observe a factor much smaller than N, then try "process parallelization". Of course, this makes only sense if the extra effort for implementing this is acceptable.
I would not be astonished if you make the same observation as me.