This is a production instance (EC2) we are managing for a client. Downtime already 5 hours. I am screwed.
The Story
This morning I found one of our clients mysqld process in killed state. I did sudo service mysql start
and got it up and running.
The log informed that this was happening due to table crash. After going through some references on the internet, I executed the following commands:
# mysqld was running
$ mysqlrepair -A --auto-repair
$ mysqlrepair -A -o
# By the time the following command was executed, mysqld had stopped
$ mysqlrepair -A -c
Ever since, the following commands refuse to start mysqld
$ sudo service mysql start
start: Job failed to start
$ sudo mysqld --verbose
$ ps aux | grep mysql
ubuntu 24124 0.0 0.1 8108 936 pts/0 S+ 04:07 0:00 grep --color=auto mysql
$ sudo mysqld_safe
131130 04:09:02 mysqld_safe Logging to '/var/log/mysql/error.log'.
131130 04:09:02 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
131130 04:09:02 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
There is nothing of the current timestamp in /var/log/mysql/error.log.
$ dmesg | grep mysql
[776212.640137] init: mysql main process (24070) terminated with status 1
[776212.640189] init: mysql main process ended, respawning
[776213.629832] init: mysql post-start process (24071) terminated with status 1
[776213.650730] type=1400 audit(1385784344.772:92): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=24094 comm="apparmor_parser"
[776213.745904] init: mysql main process (24098) terminated with status 1
[776213.745957] init: mysql respawning too fast, stopped
The configuration file is: http://pastebin.com/hbvLGqEM
Best Answer
The problem was solved by adding to my.cnf:
Reference Mike's answer:
https://askubuntu.com/questions/127264/cant-start-mysql-mysql-respawning-too-fast-stopped