How to benchmark concurrent key-value stores

I have run up against the GIL in server side programming in almost every instance where I need something to scale to millions of concurrent users on multiple core machines.

Python is great for command line tools and things that don't need true concurrency to extract every last bit of performance from a given piece of hardware.

But for things that really need to squeeze everything out of something like a Sun T2000, you don't want to write anything in Python, it will be a operational maintenance nightmare running 32 separate processes and trying to management them all.

I abandoned Twisted in favor of Erlang a few years ago, Python just doesn't cut it in the large scale concurrency space. The transparent distributed nature of Erlang means it scales horizontally as well as vertically.

There's one thing that you can do concisely and efficiently in Java that you can't in Scala: enumerations. For everything else, even for constructs that are slow in Scala's library, you can get efficient versions working in Scala.

So, for the most part, you don't need to add Java to your code. Even for code that uses enumeration in Java, there's often a solution in Scala that is adequate or good -- I place the exception on enumerations that have extra methods and whose int constant values are used.

As for what to watch out for, here are some things.

• If you use the enrich my library pattern, always convert to a class. For example:

  // WRONG -- the implementation uses reflection when calling "isWord"
  implicit def toIsWord(s: String) = new { def isWord = s matches "[A-Za-z]+" }
  
  // RIGHT
  class IsWord(s: String) { def isWord = s matches "[A-Za-z]+" }
  implicit def toIsWord(s: String): IsWord = new IsWord(s)
  

• Be wary of collection methods -- because they are polymorphic for the most part, JVM does not optimize them. You need not avoid them, but pay attention to it on critical sections. Be aware that for in Scala is implemented through method calls and anonymous classes.

• If using a Java class, such as String, Array or AnyVal classes that correspond to Java primitives, prefer the methods provided by Java when alternatives exist. For example, use length on String and Array instead of size.

• Avoid careless use of implicit conversions, as you can find yourself using conversions by mistake instead of by design.

• Extend classes instead of traits. For example, if you are extending Function1, extend AbstractFunction1 instead.

• Use -optimise and specialization to get most of Scala.

• Understand what is happening: javap is your friend, and so are a bunch of Scala flags that show what's going on.

• Scala idioms are designed to improve correctness and make the code more concise and maintainable. They are not designed for speed, so if you need to use null instead of Option in a critical path, do so! There's a reason why Scala is multi-paradigm.

• Remember that the true measure of performance is running code. See this question for an example of what may happen if you ignore that rule.