Linux – Enable HugePages on Linux for multiple services/processes

javalinuxmemoryMySQL

On one of my Debian machines I'm running a service on a JVM and a MySQL instance. I have enable the HugePages following various online guides but I'm having a problem.

Here is my configuration:

vm.nr_hugepages = 2816
vm.nr_overcommit_hugepages = 128
vm.hugetlb_shm_group = 1002
kernel.shmmax = 5905580032
kernel.shmall = 1441792

UPDATE: My machine is a guest VM on top of VMWare ESX, with assigned 2 CPUs and 6GB of RAM. I reserved 512MB or RAM for the system and the remaining (5.5GB, 5905580032bytes as seen in kernel.shmmax) is assigned 3GB to the JVM and 2.5GB to MySQL. Therefore I reserved 5.5GB / 2MB = 2816 number of huge pages

The access to the memory is allowed for the group "services" (gid=1002). MySQL and the JVM run with different users that are both members of this group "services", as primary group:

uid=106(mysql) gid=1002(services) groups=1002(services),111(mysql)
uid=1001(java) gid=1002(services) groups=1002(services),1003(java)

HugePages are enabled and they work. The problem is that only one application at the time seems to be able to use them! If I first start mysql, then the jvm will not be able to use it and vice versa.

service mysql start -> OK
service java start -> Fallback to normal memory

or

# service java start -> OK
# service mysql start -> Failure, can't allocate memory

How can I do it? Am I mistaken somewhere? Can it even be done?

There are no requirements to create a separate machine for Java and for MySQL, therefore I'm keen to keep both services on the same machine.

UPDATE: At the moment I'm doing some memory profiling to decide if it'd better to allocate the HugePages to the JVM or to MySQL. Or is there a quick answer to this?

Thank you

Best Answer

You dont mention your distro/kernel version, but it may be worth noting that recent kernels (2.6.38+) have a feature called transparent hugepages.

You can tell if it is enabled using:

cat /sys/kernel/mm/transparent_hugepage/enabled

It pretty much takes all this burden managing and preallocating hugepages of you.

You can see how much memory is allocated using which page size by running

cat /proc/meminfo | grep AnonHugePages

or

egrep 'trans|thp' /proc/vmstat

per process usage

grep -e AnonHugePages  /proc/*/smaps | awk  '{ if($2>4) print $0} ' |  awk -F "/"  '{print $0; system("ps -fp " $3)} '

Last three commands taken from RHEL6 support

Interesting info on Transparent HugePages and JVM.

Related Topic