Centos running Apache Tomcat keep getting “java.net.SocketException: Too many open files”

centosfilesperformancetomcat

We're running Apache Tomcat 7.0.41 on CentOS 6 with java version "1.7.0_21". We were getting a lot of too many open files errors so I did some research. The consensus was that it was to to with the number of open files. So I did the following:

  • Increased max files in /etc/security/limits.conf

    soft     nofile         100000
hard     nofile         100000

  • Rebooted the server

  • Checked the limits were valid for the user which was to run the process

    [app_admin@xxx ~]$ ulimit -Hn
100000
[app_admin@xxx ~]$ ulimit -Sn
100000

  • Monitored open files on the server using the lsof command

What I observed was when the total open files reached circa 13000 and tomcat had around 4500 open files the error reappeared.

I am confused. I thought it would have resolved the problem but clearly I don't fully understand the root cause and also how to set the parameter correctly.

To (maybe) help I have not modified the server.xml file for Tomcat (although I'm tempted). I don't want to start fiddling with that and make things worse.

I'm more than happy to share any more information if someone can give me some hints on where to start looking.

Best Answer

I had a similar problem with Tomcat 6 and Java 6, but my /etc/security/limits.conf looks like

tomcat           soft    nproc   2047
tomcat           hard    nproc   16384
tomcat           soft    nofile  1024
tomcat           hard    nofile  65536

Where tomcat is the user tomcat service run as.

Related Topic