I've been developing concurrent systems for several years now, and I have a pretty good grasp on the subject despite my lack of formal training (i.e. no degree). There's a few new languages that have become popular to at least talk about lately that are designed to make concurrency easier such as Erlang and Go. It appears that their approach to concurrency echoes my own experience as to how to make systems scalable and take advantage of multiple cores/processors/machines.
However, I find that there are very few tools to help visualize what you intend to do, and verify that you are at least close to your original vision. Debugging concurrent code can be a nightmare with languages that are not designed for concurrency (like C/C++, C#, Java, etc.). In particular, it can be near impossible to recreate conditions that happen readily on one system in your development environment.
So, what are your approaches to designing a system to deal with concurrency and parallel processing? Examples:
- How do you figure out what can be made concurrent vs. what has to be sequential?
- How do you reproduce error conditions and view what is happening as the application executes?
- How do you visualize the interactions between the different concurrent parts of the application?
I have my own answers for some of these, but I'd also like to learn a bit more.
Edit
So far we have a lot of good input. Many of the articles linked to are very good, and I've already read some of them.
My personal experience with concurrent programming leads me to believe you need a different mindset than you do with sequential programming. The mental divide is probably as wide as the difference between object oriented programming and procedural programming. I'd like this set of questions to focus more on the thought processes necessary (i.e. theory) to systematically approach the answers. When providing more concrete answers, it helps to provide an example–something you went through personally.
Goal for the Bounty
Don't tell me what I should do. I already have that under control. Tell me what you do. Tell me how you solve these problems.
Best Answer
Many of best programmers I know didn't finish the University. As for me I studied Philosophy.
yes
we usually start with a 1000 miles high metaphor to clarify our architecture to ourselves (firstly) and to others (secondly).
When we faced that problem, we always found a way to limiting the visibility of concurrent objects to non concurrent ones.
Lately I discovered Actors in scala and I saw that my old solutions were a kind of "miniactors", much less powerful than scala ones. So my suggestion is to start from there.
Another suggestion is to skip as many problems as possible: for example we use centralised cache (terracotta) instead of keeping maps in memory, using inner class callbacks instead of synchronised methods, sending messages instead of writing shared memory etc.
With scala it's all much easier anyway.
No real answer here. We have some unit test for concurrency and we have a load test suite to stress the application as much as we can.
Again no real answer: we design our Metaphor on the whiteboard and we try to make sure there are no conflicts on the architectural side.
For Arch here I mean the Neal Ford's definition: Sw Architecture is everything that will be very hard to change later.
Maybe but for me it's simply impossible to think in a parallel way, so better design our software in a way that doesn't require parallel thinking and with clear guardrails to avoid crashes between concurrency lanes.