Issues with time slicing

Will my application magically see and use multiple cores when run on a multi-core processor (because everything is managed either by the operating system or by the standard thread library), or do I have to modify my code to be aware of the multiple cores?

Simple answer: Yes, it will usually be managed by the operating system or threading library.

The threading subsystem in the operating system will assign threads to processors on a priority basis (your option 1). In other words, when a thread has finished executing for its time allocation or blocks, the scheduler looks for the next highest priority thread and assigns that to the CPU. The details vary from operating system to operating system.

That said, options 2 (managed by programming language) and 3 (explicitly) exist. For example, the Tasks library and async/await in recent versions of .Net give the developer a much easier way to write parallelizable (i.e. that can run concurrently with itself) code. Functional programming languages are innately parallelizable and some runtimes will run different parts of the program in parallel if possible.

As for option 3 (explicitly), Windows allows you to set the thread affinity (specifying which processors a thread can run on). However, this is usually unnecessary in all but the fastest, response-time critical systems. Effective thread to processor allocation is highly hardware dependent and is very sensitive to other applications running concurrently.

If you want to experiment, create a long running, CPU intensive task like generating a list of prime numbers or creating a Mandelbrot set. Now create two threads in your favorite library and run both threads on a multi-processor machine (in other words, just about anything released in the last few years). Both tasks should complete in roughly the same time because they are run in parallel.

One of the main reasons for using threads even in a completely CPU-bound process is to allow interaction or update progress output via a UI of some sort. While running the calculation in the foreground without allowing interaction would be more efficient, the slight cost of task switching and handling UI events usually outweighs the penalty of forcing a user to wait indefinitely for a process.

Another case where running multiple threads on a single core would be more efficient is when a multi-stage process involves one stage that produces a lot of temporary results that are processed by a later stage. Running these stages in series might exhaust available memory. Running them in parallel allows the second stage to free up memory as it processes results.

Finally, as we add more caches to our architectures, threads can often become idle when a cache miss occurs while accessing memory. This gives some time for another thread to activate and do some work.

The main benefits of multi-threading over multi-processing include

• shared memory
• less overhead for context-switching
• easier abstraction

Most modern operating systems provide some method of sharing memory across processes, too. Even still, allowing the OS or virtual machine to shift threads across multiple processors when available is very appealing. For example, when you move a multi-threaded Java program from a single-core machine to one with multiple cores, the JVM will automatically make use of those other cores.