With a simple search in amazon one can see that the modern approach for parallel programming is to use your graphic card. However I am still a little bit skeptical about it. My last computer has an 8 core CPU which I need is enough for basic all my parallel needs, if I need more I will probably use MPI through a network using my old machines. All in all, Why and/or when should I use CUDA or another method which uses my graphic card instead of traditional methods like pthreads, java threads, boost threads or the new C++ 11 threads? What about using processes?
Parallel Programming – Graphics Card vs Traditional Methods
parallelism
Best Answer
Maybe because even a mid-range Graphics card is a couple of orders of magnitude faster than an 8 core CPU at the tasks it is very good at, which is highly parallel algorithms?
8 Core CPU == 8 threads minus any OS threads that are available for your Application ( and that is only if you actually have 8
REALcores, Hyper Threading doesn't count! )512 Stream GPU = 512 Streams for your application to use.
Also the memory bandwidth on modern graphics cards are at least twice as wide and a few orders of magnitude faster than even the fastest chipset available for a general purpose CPU.
The question is confusing multi-threading with parallel concurrency which are not always used together.
GPUs aren't multi-threaded in the same way a general purpose CPU is.
Stream != ThreadThat is they don't execute multiple instructions per stream, but streams can share data that they work on, which general purpose CPUs can't do well, at least not without compromising in space over time issues. It really is comparing Apples and Oranges.