How to use AWS Batch

The root partition of an EC2 instance is taken directly from the AMI you're launching with, and it is the same size as the AMI itself. If you have a 30GB AMI, you get a 30GB root partition.

The "instance storage" described on that page comes as one or more separate scratch partitions. I don't have a running instance in front of me, but the partitions will typically show up as /dev/sdb, /dev/sdc, etc. I believe that the first is formatted as ext3 and mounted at /mnt; you have to mount (and possibly format) the other partitions yourself.

Edit: Here's what I get if I start up a Large instance using the ami-fd4aa494 AMI from http://uec-images.ubuntu.com/releases/lucid/release/.

ubuntu@ip-10-212-181-187:~$ mount | tail -n 1
/dev/sdb on /mnt type ext3 (rw)
ubuntu@ip-10-212-181-187:~$ df -h | tail -n 1
/dev/sdb              414G  199M  393G   1% /mnt
ubuntu@ip-10-212-181-187:~$ sudo blockdev --getsize64 /dev/sdb
450934865920
ubuntu@ip-10-212-181-187:~$ sudo blockdev --getsize64 /dev/sdc
450934865920

You make one up!

Putting any string < 64 chars will work. The describe call simply gives you back the client-token string used to create the instance

The idea then is then you handle failures like this.

1. Generate the client Token i.e. "Bobs instance"
2. Make successful run instance call with client-token
3. Something goes wrong on the client i.e. Script fails or times out
4. Your code starts again and generates the same client token( the trick is making sure this happens)
5. You get an a success but you actually get back the same response with same reservation-Id you got the first time you made the request and no second instance is created

I suggest reading http://docs.aws.amazon.com/AWSEC2/latest/APIReference/Run_Instance_Idempotency.html for more info on idempotency.

It's been ages since I coded this but I believe when I did it I had my workers use the SQS message ID as the client-token, so if my workers failed, or sqs delivered twice, I wouldn't leak instances.