I'm having issues with a JRuby (rails) app running in tomcat. Occasionally page requests can take up to a minute to return (even though the rails logs processed the request in seconds so it's obviously a tomcat issue).
I'm wondering what settings are optimal for the java heap size. I know there's no definitive answer, but I thought maybe someone could comment on my setup.
I'm on a small EC2 instance which has 1.7g ram. I have the following JAVA_OPTS:
-Xmx1536m -Xms256m -XX:MaxPermSize=256m -XX:+CMSClassUnloadingEnabled
My first thought is that Xmx is too high. If I only have 1.7gb and I allocated 1.5gb to java, i feel like I'll get a lot of paging. Typically my java process shows (in top) 1.1g res memory and 2g virtual.
I also read somewhere that setting the Xms and Xmx to the same size will help as it eliminates time spend on memory allocation.
I'm not a java person but I've been tasked with figuring out this problem and I'm trying to find out where to start. Any tips are greatly appreciated!!
update
I've started analyzing the garbage collection dumps using -XX:+PrintGCDetails
When i notice these occasional long load times, the gc logs go nuts. the last one I did (which took 25s to complete) I had gc log lines such as:
1720.267: [GC 1720.267: [DefNew: 27712K->16K(31104K), 0.0068020 secs] 281792K->254096K(444112K), 0.0069440 secs]
1720.294: [GC 1720.294: [DefNew: 27728K->0K(31104K), 0.0343340 secs] 281808K->254080K(444112K), 0.0344910 secs]
about 300 of them on a single request!!! Now, I don't totally understand why it's always GC'ng from ~28m down to 0 over and over.
Best Answer
While I haven't run any JRuby apps on Tomcat, I have run ColdFusion apps on varied J2EE app servers, and I also have had similar issues.
In these FAQs, you'll see that SOracle says that on 32-bit Windows, you'll be limited to a max heap size of 1.4 to 1.6 GB. I never was able to get it stable that high, and I suspect you're running a similar configuration.
My guess is that your requests are taking a long time to run b/c with a heap size that high, the JVM has allocated more physical memory than Windows had to give, and thus Windows spends a lot of time swapping pages in and out of memory to disk so it can provide the required amount of memory to the JVM.
My recommendation, although counter-intuitive, would be that you actually lower the max heap size to somewhere around 1.2 GB. You can raise the min size as well, if you notice that there are slow-downs in the app's request processing while the JVM has to ask Windows for more memory to increase the size of its heap as it fills with uncollected objects.