Not almost, but all modern CPUs have multiple cores, yet multithreading isn't really that common. Why to have these cores then? To execute several sequential programs at the same time? Well, when calculations are complex (rendering, compiling), the program is definitely made to use advantage of multiple cores. But for other tasks a single core is enough?
I understand that multi-threading is hard to implement and has drawbacks if number of threads is less than expected. But not using these idle cores seems so irrational.
Multithreading – Why Multithreading Is Not Used Everywhere
cpumultithreading
Related Solutions
You have multiple cores/procesors, use them
Async is best for doing heavy IO bound processing but what about heavy CPU bound processing?
The problem arises when single-threaded code blocks (ie gets stuck) on a long-running process. For instance, remember back when printing a word processor document would make the whole application freeze until the job was sent? Application freezing is a side-effect of a single-threaded application blocking during a CPU-intensive task.
In a multi-threaded application, CPU-intensive tasks (ex a print job) can be sent to a background worker thread thereby freeing up the UI thread.
Likewise, in a multi-process application the job can be sent via messaging (ex IPC, sockets, etc) to a subprocess designed specifically to process jobs.
In practice, async and multi-threaded/process code each have their benefits and drawbacks.
You can see the trend in the major cloud platforms, as they will offer instances specialized for CPU bound processing and instances specialized for IO bound processing.
Examples:
- Storage (ex Amazon S3, Google Cloud Drive) is CPU bound
- Web Servers are IO bound (Amazon EC2, Google App Engine)
- Databases are both, CPU bound for writes/indexing and IO bound for reads
To put it into perspective...
A webserver is a perfect example of a platform that is strongly IO bound. A multi-threaded webserver that assigns one thread per connection doesn't scale well because every thread incurs more overhead due to the increased amount of context switching and thread locking on shared resources. Whereas an async webserver would use a single address space.
Likewise, an application specialized for encoding video would work much better in a multi-threaded environment because the heavy processing involved would lock the main thread until the work was done. There are ways to mitigate this but it's much easier to have a single thread managing a queue, a second thread managing cleanup, and a pool of threads managing the heavy processing. Communication between threads happens only when tasks are assigned/completed so thread-locking overhead is kept to a bare minimum.
The best application often uses a combination of both. A webapp, for instance may use nginx (ie async single-threaded) as a load balancer to manage the torrent of incoming requests, a similar async webserver (ex Node.js) to handle http requests, and a set of multi-threaded servers handle uploading/streaming/encoding content, etc...
There have been a lot of religious wars over the years between multi-threaded, multi-process, and async models. As with the most things the best answer really should be, "it depends."
It follows a the same line of thinking that justifies using GPU and CPU architectures in parallel. Two specialized systems running in concert can have a much greater improvement than a single monolithic approach.
Neither are better because both have their uses. Use the best tool for the job.
Update:
I removed the reference to Apache and made a minor correction. Apache uses a multiprocess model which forks a process for every request increasing the amount of context switching at the kernel level. In addition, since the memory can't be shared across processes, each request incurs an additional memory cost.
Multi-threading gets around requiring additional memory because it relies on a shared memory between threads. Shared memory removes the additional memory overhead but still incurs the penalty of increased context switching. In addition -- to ensure that race conditions don't happen -- thread locks (that ensure exclusive access to only one thread at a time) are required for any resources that are shared across threads.
It's funny that you say, "programmers seems to love concurrency and multi-threaded programs in general." Multi-threaded programming is universally dreaded by anybody who has done any substantial amount of it in their time. Dead locks (a bug that happens when a resource is mistakenly locked by two different sources blocking both from ever finishing) and race conditions (where the program will mistakenly output the wrong result randomly due to incorrect sequencing) are some of the most difficult to track down and fix.
Update2:
Contrary to the blanket statement about IPC being faster than network (ie socket) communications. That's not always the case. Keep in mind that these are generalizations and implementation-specific details may have a huge impact on the result.
One of the main reasons for using threads even in a completely CPU-bound process is to allow interaction or update progress output via a UI of some sort. While running the calculation in the foreground without allowing interaction would be more efficient, the slight cost of task switching and handling UI events usually outweighs the penalty of forcing a user to wait indefinitely for a process.
Another case where running multiple threads on a single core would be more efficient is when a multi-stage process involves one stage that produces a lot of temporary results that are processed by a later stage. Running these stages in series might exhaust available memory. Running them in parallel allows the second stage to free up memory as it processes results.
Finally, as we add more caches to our architectures, threads can often become idle when a cache miss occurs while accessing memory. This gives some time for another thread to activate and do some work.
The main benefits of multi-threading over multi-processing include
- shared memory
- less overhead for context-switching
- easier abstraction
Most modern operating systems provide some method of sharing memory across processes, too. Even still, allowing the OS or virtual machine to shift threads across multiple processors when available is very appealing. For example, when you move a multi-threaded Java program from a single-core machine to one with multiple cores, the JVM will automatically make use of those other cores.
Best Answer
The proliferation of multi-core CPUs is predominantly driven by supply, not by demand.
You're right that many programmers don't bother decomposing their systems so that they can profit from multiple threads of execution. Therefore the customer sees a benefit mainly from OS multiprogramming rather than program multi-thread execution.
The reason CPU vendors create more and more cores is that the traditional way of increasing performance - increasing clock speed - has run into fundamental limitations of physics (both quantum effects and thermal problems). To keep producing chips that can be credibly sold as offering more compute power than last year's chips, they put more and more independent cores into them, trusting that OS multiprogramming and increasing use of multi-threading will catch up and yield actual rather than just nominal gains.
But make no mistake, both designing and exploiting multi-core CPUs is a lot harder than just running the same code faster. Both programmers and chip manufacturers would very much like to just keep increasing the speed of their chips; the trends towards parallelization is largely a matter of necessity, not preference.