On one of my Debian machines I'm running a service on a JVM and a MySQL instance. I have enable the HugePages following various online guides but I'm having a problem.
Here is my configuration:
vm.nr_hugepages = 2816
vm.nr_overcommit_hugepages = 128
vm.hugetlb_shm_group = 1002
kernel.shmmax = 5905580032
kernel.shmall = 1441792
UPDATE: My machine is a guest VM on top of VMWare ESX, with assigned 2 CPUs and 6GB of RAM. I reserved 512MB or RAM for the system and the remaining (5.5GB, 5905580032bytes as seen in kernel.shmmax
) is assigned 3GB to the JVM and 2.5GB to MySQL. Therefore I reserved 5.5GB / 2MB = 2816 number of huge pages
The access to the memory is allowed for the group "services" (gid=1002). MySQL and the JVM run with different users that are both members of this group "services", as primary group:
uid=106(mysql) gid=1002(services) groups=1002(services),111(mysql)
uid=1001(java) gid=1002(services) groups=1002(services),1003(java)
HugePages are enabled and they work. The problem is that only one application at the time seems to be able to use them! If I first start mysql, then the jvm will not be able to use it and vice versa.
service mysql start -> OK
service java start -> Fallback to normal memory
or
# service java start -> OK
# service mysql start -> Failure, can't allocate memory
How can I do it? Am I mistaken somewhere? Can it even be done?
There are no requirements to create a separate machine for Java and for MySQL, therefore I'm keen to keep both services on the same machine.
UPDATE: At the moment I'm doing some memory profiling to decide if it'd better to allocate the HugePages to the JVM or to MySQL. Or is there a quick answer to this?
Thank you
Best Answer
You dont mention your distro/kernel version, but it may be worth noting that recent kernels (2.6.38+) have a feature called transparent hugepages.
You can tell if it is enabled using:
It pretty much takes all this burden managing and preallocating hugepages of you.
You can see how much memory is allocated using which page size by running
or
per process usage
Last three commands taken from RHEL6 support
Interesting info on Transparent HugePages and JVM.