I've been developing concurrent
systems for several years now, and I
have a pretty good grasp on the
subject despite my lack of formal
training (i.e. no degree).
Many of best programmers I know didn't finish the University.
As for me I studied Philosophy.
C/C++, C#, Java, etc.). In particular,
it can be near impossible to recreate
conditions that happen readily on one
system in your development
environment.
yes
How do you figure out what can be made concurrent vs. what has to be
sequential?
we usually start with a 1000 miles high metaphor to clarify our architecture to ourselves (firstly) and to others (secondly).
When we faced that problem, we always found a way to limiting the visibility of concurrent objects to non concurrent ones.
Lately I discovered Actors in scala and I saw that my old solutions were a kind of "miniactors", much less powerful than scala ones. So my suggestion is to start from there.
Another suggestion is to skip as many problems as possible: for example we use centralised cache (terracotta) instead of keeping maps in memory, using inner class callbacks instead of synchronised methods, sending messages instead of writing shared memory etc.
With scala it's all much easier anyway.
How do you reproduce error conditions and view what is happening
as the application executes?
No real answer here. We have some unit test for concurrency and we have a load test suite to stress the application as much as we can.
How do you visualize the interactions between the different
concurrent parts of the application?
Again no real answer: we design our Metaphor on the whiteboard and we try to make sure there are no conflicts on the architectural side.
For Arch here I mean the Neal Ford's definition: Sw Architecture is everything that will be very hard to change later.
programming leads me to believe you
need a different mindset than you do
with sequential programming.
Maybe but for me it's simply impossible to think in a parallel way, so better design our software in a way that doesn't require parallel thinking and with clear guardrails to avoid crashes between concurrency lanes.
I have seen reference to using ring buffers, and controlled access the location pointers to eliminate or reduce the need for locks. It does not eliminate the need for waits though, and should only work then a ring buffer has one writer and one reader. In your case you would need at least two buffers.
The mechanism as I understand it is:
Depending on the wait time used this can either add latency for new items in an empty queue, or burn a lot of CPU cycles in the wait loop. The order of updating the pointer and strict separation of write access is critical for this to work. The writer is only allowed to write the ring buffer entry before incrementing the pointer to make it available to the reader.
Best Answer
Good concurrency requires a lot more than throwing a few threads in an application and hoping for the best. There's a range in how concurrent a program can be going from embarrassingly parallel to pure sequential. Any given program can use Amdahl's law to express how scalable a problem or algorithm is. A couple qualifications for a embarrassingly parallel application would be:
There are other qualifications, but with just these two we can understand why games in particular are not as easy as you might think to take advantage of multiple cores. For one, the model of the world that will be rendered has to be shared as different functions calculate physics, movement, apply artificial intelligence etc. Second, each frame of this game model has to be rendered on screen with a graphics card.
To be fair, many game makers use game engines that are produced by third parties. It took a while, but these third party game engines are now much more parallel than they used to be.
There are bigger architectural challenges in dealing with effective concurrency
Concurrency can take many forms, from running tasks in the background to a full architectural support for concurrency. Some languages give you very powerful concurrency features such as ERLANG, but it requires you to think very differently about how you construct your application.
Not every program really needs the complexity of full multicore support. One such example is tax software, or any form driven application. When most of your time is spent waiting on the user to do something, the complexity of multithreaded applications are just not that useful.
Some applications lend themselves to a more embarrassingly parallel solution, such as web applications. In this case, the platform starts out embarrassingly parallel and it's up to you not have to impose thread contention.
The bottom line:
Not all applications are really hurt by not taking advantage of multiple threads (and thus, cores). For the ones that are hurt by that, sometimes the computations are not friendly to parallel processing or the overhead to coordinate it would make the application more fragile. Unfortunately, parallel processing is still not as easy as it should be to do well.