I'd recommend something that can do incremental backups, because that way you can go back to files that were changed several days ago. We use dirvish for this (it's available from the ubuntu repos) - basically it's a wrapper for rsync written in perl that makes it easier to setup a system to run regular incremental backups, and then expire them based on given rules.
On our system we keep backups for 7 days, but keep the first backup of the week for 1 month, and the first backup of the month for one year, so usually you can find files that were modified and deleted some time ago. For things that are small, but important, we have much longer expiry times.
As for what you should backup, imo if you have the space back up everything unless you're sure you don't need to back up. With incremental rsync backups you're only copying over the files that have changed since the last backup - I'd rather have an extra 3gb of static data belonging to the operating system on my backup drive than find it was missing one important file when I needed to recover it.
At a minimum I'd backup the entire contents of /etc/, and most of /var depending on what stuff was running on the server.
If your mysql database is fairly small then your current approach is probably acceptable, but since you need to lock the database to generate the dump this isn't practical on large databases. Because of this lots of people run a slave mysql server, which is just a read only copy of the main one on the backup server, and use this to generate backups.
The mysql documentation on how to do this is fairly good, I worked through it the first time in an evening, without much previous mysql experience.
We have our slave mysql server tied into dirvish, so each day it generates a dump of each individual database, as well as an entire copy of /var/lib/mysql (it's stops the mysql server before doing this)
I took Garth's suggestion and ran the script with sh -x. I found out there are permission issues when accessing the mysql.sock file. So I left the old one in the conf file, ran it again and ta-da it works and runs from the new location!
Thanks Gareth for showing me the way. I gave you a +1 on the comment but you did not post an answer.
Hopefully, this question will help some other people trying to do the same.
Best Answer
doing a backup of /var/lib/mysql is not a good idea, because the mysql daemon needs to be stopped before.
use tools like mysqldump to dump the databases into a file in e.g. /var/backups/mysql/ or whereever and backup these files
the dump of the databases can be started by the user creating the backup, so there will be no problems. but think of removing rights on the folder for group and world if you do not need them