Solaris 10- high kernel utilization (%sys)

Please note that Solaris 11 has not been released but that should happen until the end of this year (or sooner).

With that in mind, your options are Solaris 10 and Solaris 11 Express. Both are supported in production and will require you to buy a license. So I'm assuming that's not a problem.

Since you're asking which one to use, I'm also assuming your company doesn't have a hard policy in place. Solaris 11 Express is the continuation of the OpenSolaris project and offers a lot of improvements (there are tons of documents detailing it) and it has access to a much bigger list of current software ready to be installed.

The problem with Solaris 11 Express is that the path to Solaris 11 is not clear, although it shouldn't be hard since both will be based on IPS. People have upgraded from OpenSolaris 2009.6 (b111) to Solaris 11 Express (b151a) without much trouble but that still means you'll have to stop in the near future to upgrade to Solaris 11 "proper". In some environment that's not desired but I'm guessing in your web server it should be acceptable.

It's very hard to decide this for you but, unless you've more rigid software requirements that mandate you use Solaris 10 (in that case you wouldn't be asking here), I advise you to install and test Solaris 11 Express. Solaris 10 is nowhere near it's end of life but it's nevertheless sooner than Solaris 11 will be. Solaris 11 Express also offers a more pleasant experience for people used to Linux/BSD environments.

Do you VM's have the same number of processor's as the host system? if so this is a bad thing, and it can prevent the scheduler from working properly. IE if you have an 8 core system then no system on that box should have 8 cores assigned to it. You can have 20 VM's with 4 Cores assigned and that is not a problem, but 1 box with 8 cores assigned can cause problems under load.