Java – How Java multi-threaded program is able to use multiple CPU cores

Does every thread create their own instance of the JVM to handle their particular execution?

No. They execute in the same JVM so that (for example) they can share objects and class attributes.

If not then does the JVM have to have some way to schedule which thread it will handle next

There are two kinds of thread implementation in Java. Native threads are mapped onto a thread abstraction which is implemented by the host OS. The OS takes care of native thread scheduling, and time slicing.

The second kind of thread is "green threads". These are implemented and managed by the JVM itself, with the JVM implementing thread scheduling. Java green thread implementations have not been supported by Sun / Oracle JVMs since Java 1.2. (See Green Threads vs Non Green Threads)

If so wouldn't this render the multi-threaded nature of Java useless since only one thread can be ran at a time?

We are talking about green threads now, and this is of historic interest (only) from the Java perspective.

• Green threads have the advantage that scheduling and context switching are faster in the non-I/O case. (Based on measurements made with Java on Linux 2.2; http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.8.9238)

• With pure green threads, N programming language threads are mapped to a single native thread. In this model you don't get true parallel execution, as you noted.

• In a hybrid thread implementation, N programming language threads are mapped onto M native threads (where N > M). In this model, the in-process thread scheduler is responsible for the green thread to native thread scheduling AND you get true parallel execution (if M > 1); see https://stackoverflow.com/a/16965741/139985.

But even with the pure green threads, you still get concurrency. Control is switched to another threads a thread blocks on an I/O operation, whick acquiring a lock, and so on. Furthermore, the JVM's runtime could implement periodic thread preemption so that a CPU intensive thread doesn't monopolize the (single) core to the exclusion of other threads

Each Thread will be given a native OS Thread to RUN which can run on a different CPU but since Java is interpreted these threads will require to interact with the JVM again and again to convert the byte code to machine instructions ? Am I right ?

You are mixing two different things; JIT done by the VM and the threading support offered by the VM. Deep down inside, everything you do translates to some sort of native code. A byte-code instruction which uses thread is no different than a JIT'ed code which accesses threads.

If yes, than for smaller programs Java Threads wont be a big advantage ?

Define small here. For short lived processes, yes, threading doesn't make that big a difference since your sequential execution is fast enough. Note that this again depends on the problem being solved. For UI toolkits, no matter how small the application, some sort of threading/asynchronous execution is required to keep the UI responsive.

Threading also makes sense when you have things which can be run in parallel. A typical example would be doing heavy IO in on thread and computation in another. You really wouldn't want to block your processing just because your main thread is blocked doing IO.

Once the Hotspot compiles both these execution paths both can be as good as native Threads ? Am I right ?

See my first point.

Threading really isn't a silver bullet, esp when it comes to the common misconception of "use threads to make this code go faster". A bit of reading and experience will be your best bet. Can I recommend getting a copy of this awesome book? :-)

@Sanjay: Infact now I can reframe my question. If I have a Thread whose code has not been JIT'd how does the OS Thread execute it ?

Again I'll say it, threading is a completely different concept from JIT. Let's try to look at the execution of a program in simple terms:

java pkg.MyClass -> VM locates method to be run -> Start executing the byte-code for method line by line -> convert each byte-code instruction to its native counterpart -> instruction executed by OS -> instruction executed by machine

When JIT has kicked in:

java pkg.MyClass -> VM locates method to be run which has been JIT'ed -> locate the associated native code for that method -> instruction executed by OS -> instruction executed by machine

As you can see, irrespective of the route you follow, the VM instruction has to be mapped to its native counterpart at some point in time. Whether that native code is stored for further re-use or thrown away if a different thing (optimization, remember?).

Hence to answer your question, whenever you write threading code, it is translated to native code and run by the OS. Whether that translation is done on the fly or looked up at that point in time is a completely different issue.