Linux – way to set ulimit for all root processes

These limits came from a time where multiple "normal" users (not apps) would share the server, and we needed ways to protect them from using too many resources.

They are very low for high performance servers and we generally set them to a very high number. (24k or so) If you need higher numbers, you also need to change the sysctl file-max option (generally limited to 40k on ubuntu and 70k on rhel) .

Setting ulimit:

# ulimit -n 99999

Sysctl max files:

#sysctl -w fs.file-max=100000

Also, and very important, you may need to check if your application has a memory/file descriptor leak. Use lsof to see all it has open to see if they are valid or not. Don't try to change your system to work around applications bugs.

This is almost exactly why Linux control groups (cgroups) were invented. This allows a group of processes (for example, all the processes decending from your initial interactive login) to be treated as a single entity for purposes of resource control -- such as limiting the total amount of memory consumed (or cpu resources, etc).

Unfortunately...

Cgroups are relatively new, and convenient support for them is largely missing in most distributions.