Linux – How to automate website deployments to production including minifying CSS/JS

deploymentfabriclinuxrsyncssh

Currently I'm deploying my PHP website from staging to production using the following manual process:

Minify CSS and JavaScript files using online YUI compressor tool.
Move the original CSS and JavaScript files into another directory (to back them up) then replace them with the minified CSS and JavaScript CSS files.
Start copying all the files from my staging server to my Windows PC.
Delete staging .htaccess and index files so they don't get copied to the production server (these files have specific stuff for the staging environment).
Open WinSCP and go to the web root on the production server then switch out the main index file for an 'updates in progress' one so if any visitors come to the site then they get a message saying it's down temporarily.
Then copy from my PC to the production web server using WinSCP (this overwrites all existing files on the server) and takes about 4 minutes seeing there's lots of files.
Log into the production server with SSH and run about 5 commands to set the proper permissions for various writable directories and make sure everything is still under www-root group. I think the upload process with SSH changes all the existing permissions.
Switch out the 'updates in progress' index file back to normal index.

All up this process is pretty convoluted and takes about 5-10mins and I have to do it each time I make changes to the website so it has become a chore.

Now is there any way to automate this process using some scripting tool? Or is there a nice deployment tool that people use that can do all this? My staging and production machines are Ubuntu 12.04 server based so potentially I could use bash scripting to do some of the work.

Some improvements I can see that could be made so far would be:

Add some code to my website so it detects if it's in Staging or Production then uses the original or minified CSS/JavaScript files depending on the environment. This will save me swapping the files around manually.
Use the commandline YUI compressor and run that from a script file when needed.
Use something like rsync so it only copies the changed files from my staging server to the production server not overwriting every file

One company I worked for used the Fabric command line tool for automating the deployments and doing this kind of thing. However is this a good option or is there a better tool out there?

Best Answer

Fabric would be ideal for this kind of task. If you're familiar with python you have a lot of flexibility with what you can do.

Capistrano is another option that might be useful, I have used it for Rails deployments and had no issues there. Allows you to easily run shell commands on remote hosts.

Another option might be to use Ant (python way with fabric is much nicer imo).

Are you using source control like git? You could add some of these tasks as post-commit hooks when you push it to your "production" branch.

Rsync would be the quickest and safest option. Make sure you exclude any files which should not be in production (backup files, vim .swp files, etc).

Related Solutions

Linux – Atomic copying of multiple directories

I think you are on the right track with a staging area. I'm not aware of any atomic commands, but if you stage your files and then use a script to remove the first directory and move (not copy) the second, and do it for all three directories, it should be really quick.

Alternatively, you may want to use symlinks. That way you could have roughly:

/version/22/<my-apps>
/version/22/<my-site>
/version/22/<my-media>

and deploy a /version/23 directory with the same subdirectories. Then, where the actual file would go (and again, for speed, you'll need a script), you can use a symbolic link so that when anyone goes to the latest page, they get whatever version is current (and it all happens transparently and they have no idea). The benefit of this is that your older work is still around until you decide to delete it. [Although, granted, a versioning control system is a better thing to use for keeping older work.]

You'd have to check that 1) you can run scripts (and in such a way that web users can not!), and 2) that you can use symlinks (as some web servers are configured to not follow them.)

Ssh – How to automate SSH login with password

Don't use a password. Generate a passphrase-less SSH key and push it to your VM.

If you already have an SSH key, you can skip this step… Just hit Enter for the key and both passphrases:

$ ssh-keygen -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.

Copy your keys to the target server:

$ ssh-copy-id id@server
id@server's password:

Now try logging into the machine, with ssh 'id@server', and check-in:

.ssh/authorized_keys

Note: If you don't have .ssh dir and authorized_keys file, you need to create it first

to make sure we haven’t added extra keys that you weren’t expecting.

Finally, check to log in…

$ ssh id@server

id@server:~$

You may also want to look into using ssh-agent if you want to try keeping your keys protected with a passphrase.

Best Answer

Related Solutions

Linux – Atomic copying of multiple directories

Ssh – How to automate SSH login with password

Related Topic