Run tomcat as the tomcat user. I don't think you should ever run tomcat as root. It exposes the potential for a larger security risk than running it as an unprivileged user.
Tomcat's HTTP serving is a bit lacking for some stuff, so i'd always suggest that you have a http server as a frontend. If you don't want apache, there's always Nginx. This is especially true if you wanted HTTP Basic or Digest auth.
I'd suggest that you start off with tomcat's default memory settings, then tune them as and when you need to. With no knowledge of the app's memory profile, it's difficult to give you any hard and fast figures.
If you're using Ubuntu on your vps, you can just apt-get install tomcat6 and it should work out of the "box". Even on Centos5.4, the tomcat5 package from yum works with minimal config changes.
I seriously advise that you use a lightweight server in front of your tomcat instance though. This gives you better granular control over security than Tomcat's "Security Manager"
I have Tomcat6 running on a Virtual Machine which runs my hudson CI server, I think i've given it about 256MB of RAM, and it's perfectly happy.
Apache
Check out Apache's own documentation, it goes into more detail than I could here:
http://httpd.apache.org/docs/2.0/misc/perf-tuning.html
JVM
Set your JVM's Xmx to no more than 70% (roughly) of the total free physical RAM. The reason for this is that the perm gen and JVM libraries take additional space up too - the aim is that the total process memory will never use virtual / swap memory. If you set this too high, you'll start seeing issues like "GC overhead limit exceeded".
Your GC algorithm can have a big effect on performance - make sure that you're using some form of parallel collector and not the serial 'pause, mark and sweep'. The JVM usually does this for you automically in -server
mode.
Use a tool such as JConsole or JVisual VM to inspect the GC and how much heap you're actually using, and adjust down to suit - too large a heap can impact garbage collection times.
Tomcat
As for HTTP connector threads, on a single instance of Tomcat, depending on your application you can usually up the thread count to around 600 before encountering issues - however, there's often no need for it to be quite that high - you'll just be putting more pressure on your CPU and memory.
Once you're happy with the max threads, I then set the minSpareThreads
and maxSpareThreads
relative to that. Upping the values if I know I'm gonna get hit with spikes in new connections etc.
Next up acceptCount
. This is the maximum queued connections -c onnections that spill over this setting after using up the connector threads will receive a "connection refused".
As a minor tweek, you can set enableLookups
(allow DNS hostname lookups) to false. When enabled, (slightly) adversely affects performance.
Also, check out the Tomcat Native Library, this uses native code to boost performance in certain operations (like file IO etc).
Load Testing
For basic load / performance testing, check out Apache JMeter:
http://jakarta.apache.org/jmeter/
We use it to test basic page load performance, with JMeter test scripts using hundreds of concurrent requests. You do need a fairly hefty server to run it on though (not on the same machine you're running Apache HTTPD and Tomcat).
Best Answer
The first thing I would do is reduce the likelyhood that any of those processes can take more CPU or disk IO time than the OS. I am going to assume your OS is linux.
Be sure to back up any config files before editing them.
You may be able to get some hints to the OS behavior just prior to the crash by looking at sar data.
Be sure to look for a climb in memory or CPU usage. You can have sar run more often by editing /etc/cron.d/sysstat assuming it is installed and enabled.
For each of the service accounts your processes are running as, you could add the following to /etc/security/limits.conf at the end of the file.
Then in each of the init scripts for your daemons, reduce the CPU and IO time allotted to them.
Add the following on the second line of the script to reduce CPU and IO time slices:
Restart each of your services.
Let's assume that sshd will still become unresponsive. If you install "screen", then you can have vmstat, iotop and other tools running in various screens. There are cheat sheets on using screen so I will not cover that here.
At this point, even if your services are getting out of control, you should still have the ability to ssh to the server assuming it is not triggering a panic.
You can further restrict the resources allocated to each daemon by pinning them to a specific core or CPU. This can be done with the command "taskset". man taskset for more details on its usage.
[edit] I should also add that this won't help under certain spinlock conditions. If the above does not help, you may have to run your applications in a VM and use a debug kernel or other debugging tools.