MySQL Recovery – Best Way to Recover MySQL After chmod -R 777 /

chmodMySQLpermissionsubuntu-12.04

Question:
What is the best way to recover mysql (or worst case: migrate away) from a "chmod -R 777 /" with databases intact?

System:
Ubuntu 12.04 LTS
MySQL 5.5.24
64 bit Amazon EC2 cloud server.

Background:
Attempting to recover (or at least recover data from) a system which had this done to it: 

    chmod -R 777 /

No point in worrying about the why. It was a manager with too much access and too little experience who likes to swim in deep waters. It was a pure accident on his part, not actually meaning to hit enter when he did.

I have recovered much of the system, but am really hung up on getting mysql working again. I have already worked through such pages as:

Have already done this: 

    sudo chmod 644 my.cnf
    chown mysql:mysql my.cnf

At which point attempting to start mysql: 

    sudo service mysql start

Produces this output in syslog:

    Apr 12 20:51:42 ip-10-10-25-143 kernel: [18632541.774742] type=1400 audit(1365799902.306:41): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=18210 comm="apparmor_parser"
    Apr 12 20:51:42 ip-10-10-25-143 kernel: [18632541.964496] init: mysql main process (18214) terminated with status 1
    Apr 12 20:51:42 ip-10-10-25-143 kernel: [18632541.964542] init: mysql main process ended, respawning
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632542.959796] init: mysql post-start process (18215) terminated with status 1
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.002041] type=1400 audit(1365799903.534:42): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=18238 comm="apparmor_parser"
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.098490] init: mysql main process (18242) terminated with status 1
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.098536] init: mysql main process ended, respawning
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.140706] init: mysql post-start process (18244) terminated with status 1
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.158681] type=1400 audit(1365799903.690:43): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=18258 comm="apparmor_parser"
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.285087] init: mysql main process (18262) terminated with status 1
    Apr 12 20:51:43 ip-10-10-25-143 kernel: [18632543.285133] init: mysql respawning too fast, stopped

What I read from that is mysql is terminating with a status 1, and it loops a few times trying to start up and gets stopped from doing that after too many attempts. I've looked into status 1 but haven't found solutions that seem applicable.

Best Answer

  1. Create new VM instance with the same OS version and MySQL version.
  2. Start MySQL on new VM - it should create MySQL data directory with no databases but with correct permissions - in /var/lib/mysql.
  3. Stop MySQL on new VM.
  4. Copy /var/lib/mysql on new VM to /var/lib/mysql.empty.
  5. Copy /var/lib/mysql from old VM to /var/lib/mysql on new VM.
  6. Manually set permission on all files and directories in /var/lib/mysql based on permissions on /var/lib/mysql.empty.
  7. A short prayer to a deity of choice wouldn't hurt now.
  8. Start MySQL on new VM.
  9. I'd recommend dumping all data with mysqldump -A, recreating a new, empty mysql data directory and importing this data back just in case.
  10. When you are confident that it works and all your data is intact shutdown the old VM and archive it's disk image somewhere. It is much less work and much safer to setup a new server than trying to recover from this.
Related Topic