Today I've seen MySQL/MariaDB stop running on a CentOS VPS. When I check the status I get:
# service mysql status
ERROR! MySQL is not running, but lock file (/var/lock/subsys/mysql) exists
So I remove that file and then restart the service. But then later in the day, MySQL will be offline again and the lock file will be there still. So something is happening that is causing MySQL to die but leave the lock file, which tells me its not a clean/smooth exit.
In the past when I've seen the lock file exist but the service not running it was cause the entire VPS had a sudden reboot. But this isn't the case here because my uptime is 160+ days on the VPS.
I've noticed fail2ban banning some random hack attempts on a WordPress site on the VPS today. So there has been a fair amount of traffic trying to hammer the server here and there until they get autobanned.
- How can I determine what is causing MySQL to stop randomly?
- What is the recommended method for auto-restarting MySQL? Can I use something like Supervisor?
UPDATE: After looking through my messages log I'm seeing this:
Dec 1 14:06:42 localhost kernel: Out of memory: Kill process 25063 (mysqld) score 24 or sacrifice child
Dec 1 14:06:42 localhost kernel: Killed process 25063, UID 27, (mysqld) total-vm:773716kB, anon-rss:32300kB, file-rss:48kB
So this mean that MySQL is simply being overloaded with requests (probably from httpd)?
Best Answer
The OOM killer fired as the system was out of RAM and swap - the
mysqldprocess was chosen as its victim, likely due to having a combination of high RAM use and relatively low activity on its RAM.Either adjust the available buffer pool size to make MariaDB take less RAM and not trigger the OOM killer (which might hurt performance) or give the system more RAM or swap.