Java Multithreading – Optimal Number of Threads for NIO Server

javamulti-coremultithreadingserver-side

I've built a Java NIO TCP server, it currently uses four threads.
One ServerRunnable thread that uses a selector and three worker threads.

I've been looking around for some information about this, as I've read in the past that you should only have one thread per processor core. So that's that.

But that got me wondering recently, and after a little more research I came across this thread.

Where, in the comments on the accepted answer user Donal Fellows points out the following:

Have at most one CPU-bound thread per processor allocated to the application. IO-bound threads aren't a big problem (other than the memory they consume) and it's important to remember that apps can be restricted to only use a subset of the system's CPUs; after all, it's (usually) the user's/admin's computer and not the programmer's.

With that in mind, am I correct in thinking I can safely increase the number of selector threads and worker threads in my thread pool.

My server thread reads input, processes the data into JSONObjects and then pushes them to a queue. The worker threads then take the JSONObjects from the queue, checks what type of objects they are and then pushes them to the database.
So there is very little computational work going on there. Is it safe enough for me to increase the number of threads here, as in use more ServerRunnable threads and more worker threads?
Say to double the amount of each for example?

What do I need to think about when considering something like this?

Best Answer

You have no reason to assume how many CPU cores the users system has. You development machine might run on a 4-core CPU with nothing else to do, but it could also be moved to a single-core virtual machine or a 32-core high-end server.

For that reason you should not hard-code the number of CPUs.

In Java, you can use ThreadPoolExecutor's to delegate the thread management to the JVM. You usually pass smaller work packages to the executor in form of objects which implement Runnable and leave the decision which thread processes each runnable to the thread pool. In your case, the individual JSONObjects would be such work packages.

The ThreadPoolExecutor allows you to set the minimum and maximum number of CPUs it is allowed to use. I would recommend you to make this configurable and default to Runtime.getRuntime().availableProcessors() when no configuration option is provided.

Whether or not your IO thread counts as an actual thread in regards to its CPU load depends on what it is actually doing. When it receives a high IO bandwidth and has considerable parsing work to do, it might. But we can not tell you this without profiling your application under real-world conditions.

Related Topic