R – What are some alternatives to a Workflow Engine

You could try Simple State Machine. You would have to implement access control and background timers yourself, but that shouldn't be a big deal. SSM was also built out of frustration with WF. There are some other state machine implementations on Codeplex as well. If one of them doesn't fit he bill out of the box, they are open source and should get you close enough.

I wholeheartedly agree with you about state machines in WF - they aren't testable, are too complicated, the threading model is peculiar and hard to follow, and I'm not sure a visual designer could have been more poorly conceived for designing state machines graphically. I think this may be because the state machine concept feels tacked onto the WF runtime, which was designed for sequential state machines, something WF does a much better job with, in my opinion. The problem is that state machines are really not the same animal as a sequential work flow, and should have been given a first class implementation of their own, because the warping of WF to make it seem to support them turned out to be more or less unsupportable, if not actually unusable.

Why do you regard Javaspaces as dead, beyond the fact that the Jini 2.1 release was some time ago (October 2005) ? Having used that, I'd suggest that it indicates a mature and complete technology set rather than something abandoned and defunct.

For another implementation of Javaspaces, take a look at Blitz Javaspaces. That's maintained and enhanced more regularly (latest release July 2008) and offers a more performant and manageable Javaspace implementation than the default outrigger supplied by Sun.