Docker – How to persist all Jenkins data on an EBS volume in docker

I had the same question as you, so I worked out how to do it.

First, I did this from the Ubuntu 32-bit EBS-backed ami from the US-East region, other OS's or images may work differently. However, I suspect that you should be ok, as long as you are using an ext* filesystem. It might work on other filesystems, but you'll have to figure out how to resize them on your own.

The steps are basically:

1. Attach two volumes to a running instance, the first based on the snapshot you want to shrink, and the second a blank volume having the new size you want to shrink to.

2. Check the file system of the first volume and repair any errors.

3. Shrink the file system on the first volume so it is only as big as it needs to be to hold the data.

4. Copy the file system from the first volume to the second.

5. Expand the file system on the second volume to it's maximum size.

6. Make sure everything looks good by checking the second volume for errors.

7. Take a snapshot of the second volume.

8. Create a machine image based on the snapshot of the second volume you just took.

You first need to get some information from the ami you want to shrink. In particular, you need the kernel ID and ramdisk ID, if any (the image I shrunk didn't have a ramdisk). All this information should be available from the aws management console ,in the AMI window.

The kernel ID looks like kia-xxxxxxxx, and the snapshot ID looks like snap-xxxxxxxx, and ramdisk IDs look like RIA-xxxxxxxx.

Next, launch a linux instance. I launched a Ubuntu instance. You can use a t1.micro instance if you like. It doesn't take much power to do these next steps.

After the machine is running, attach the snapshot you wrote down from the first step. In my case, I attached it to /dev/sdf

Then, create a new volume, having the size you want. In my case, I created a 5GB volume, as that's the size I wanted to shrink it to. Don't create this new volume from a snapshot. We need a new blank volume. Next, attach it to the running instance, in my case I attached it as /dev/sdg

Next, ssh into the machine but don't mount the attached volumes.

At this point, I erred on the side of paranoia, and I opted to check the file system on the large volume, just to make sure there were no errors. If you are confident that there are none, you can skip this step:

$ sudo e2fsck -f /dev/sdf

Next, I resized the file system on the large volume so that it was only as big as the data on the disk:

$ sudo resize2fs -M -p /dev/sdf

The -M shrinks it, and the -p prints the progress.

The resize2fs should tell you how large the shrunkin filesystem is. In my case, it gave me the size in 4K blocks.

We now copy the shrunkin file system to the new disk. We're going to copy the data in 16MB chunks, so we need to figure out how many 16MB chunks we need to copy. This is where that shrunken file system size comes in handey.

In my case, the shrunk file system was just over 1 GB, because I had installed a lot of other programs on the basic Ubuntu system before taking a snapshot. I probably could have gotten away with just copying the size of the file system rounded up to the nearest 16MB, but I wanted to play it safe.

So, 128 times 16MB chunks = 2GB:

$ sudo dd if=/dev/sdf ibs=16M of=/dev/sdg obs=16M count=128

I copied in blocks of 16MB because with EBS, you pay for each read and write, so I wanted to minimize the number of them as much as possible. I don't know if doing it this way did so, but it probably didn't hurt.

We then need to resize the file system we just copied to the new volume so that it uses all the available space on the volume.

$ sudo resize2fs -p /dev/sdg

Finally, check it, to make sure everything is well:

$ sudo e2fsck -f /dev/sdg

That's all we need to do in this machine, though it couldn't hurt to mount the new volume, just as a test. However, this step is almost certainly optional, as e2fsck should have caught any problems.

We now need to snapshot the new volume, and create an AMI based on it. We're done with the machine, so you can terminate it if you like.

Make sure the small volume is unmounted if you mounted it, and then take a snapshot of it. Again, you can do this in the management console.

The final step requires the commandline ec2 tools.

EDIT:

Since this answer was posted the AWS console allows you to simply right click a snapshot and select Create Image from Snapshot. You will still need to select the appropriate Kernel ID. If it does not appear on the list make sure you've selected the appropriate architecture.

We use the ec2-register application to register an AMI based on the snapshot you just took, so write down the snap-xxxxxxxx value from the snapshot you just took.

You should then use a command like:

ec2-register -C cert.pem -K sk.pem -n The_Name_of_Your_New_Image
-d Your_Description_of_This_New_AMI --kernel aki-xxxxxxxx
-b "/dev/sda1=snap-xxxxxxxx" --root-device-name /dev/sda1

You of course need to replace the kernel ID with the one you wrote down at the beginning and the snapshot ID with the one you created in the previous step. You also need to point it at your secret key (called sk.pem) above, and your x509 cert (called cert.pem). You can of course choose whatever you want for the name and description.

Hope this helps.

An AMI, as you note, is a machine image. It's a total snapshot of a system stored as an image that can be launched as an instance. We'll get back to AMIs in a second.

Lets look at EBS. Your other two items are sub-items of this. EBS is a virtual block device. You can think of it as a hard drive, although it's really a bunch of software magic to link into another kind of storage device but make it look like a hard drive to an instance.

EBS is just the name for the whole service. Inside of EBS you have what are called volumes. These are the "unit" amazon is selling you. You create a volume and they allocate you X number of gigabytes and you use it like a hard drive that you can plug into any of your running computers (instances). Volumes can either be created blank or from a snapshot copy of previous volume, which brings us to the next topic.

Snapshots are ... well ... snapshots of volumes: an exact capture of what a volume looked like at a particular moment in time, including all its data. You could have a volume, attach it to your instance, fill it up with stuff, then snapshot it, but keep using it. The volume contents would keep changing as you used it as a file system but the snapshot would be frozen in time. You could create a new volume using this snapshot as a base. The new volume would look exactly like your first disk did when you took the snapshot. You could start using the new volume in place of the old one to roll-back your data, or maybe attach the same data set to a second machine. You can keep taking snapshots of volumes at any point in time. It's like a freeze-frame instance backup that can then easy be made into a new live disk (volume) whenever you need it.

So volumes can be based on new blank space or on a snapshot. Got that? Volumes can be attached and detached from any instances, but only connected to one instance at a time, just like the physical disk that they are a virtual abstraction of.

Now back to AMIs. These are tricky because there are two types. One creates an ephemeral instances where the root files system looks like a drive to the computer but actually sits in memory somewhere and vaporizes the minute it stops being used. The other kind is called an EBS backed instance. This means that when your instances loads up, it loads its root file system onto a new EBS volume, basically layering the EC2 virtual machine technology on top of their EBS technology. A regular EBS volume is something that sits next to EC2 and can be attached, but an EBS backed instance also IS a volume itself.

A regular AMI is just a big chunk of data that gets loaded up as a machine. An EBS backed AMI will get loaded up onto an EBS volume, so you can shut it down and it will start back up from where you left off just like a real disk would.

Now put it all together. If an instance is EBS backed, you can also snapshot it. Basically this does exactly what a regular snapshot would ... a freeze frame of the root disk of your computer at a moment in time. In practice, it does two things different. One is it shuts down your instance so that you get a copy of the disk as it would look to an OFF computer, not an ON one. This makes it easier to boot up :) So when you snapshot an instance, it shuts it down, takes the disk picture, then starts up again. Secondly, it saves that images as an AMI instead of as a regular disk snapshot. Basically it's a bootable snapshot of a volume.