How to find out when an EC2 instance was shutdown

By default, EC2 instance "terminate" automatically deletes all EBS volumes that were automatically created with the instance, but this can be changed. It does not,by default, delete EBS volumes that were attached after the instance started running, and this also can be changed.

Here's an article I wrote on how to protect your important data with EBS boot instances:

Three Ways to Protect EC2 Instances from Accidental Termination and Loss of Data
http://alestic.com/2010/01/ec2-instance-locking

Note: Instance failure should not automatically delete your EBS volume. However, EBS volume failure is itself one failure mode, so make sure you are creating regular EBS snapshots. Not only does this give you a backup to rely on, but it also magically and transparently increases the reliability of the EBS volume itself.

Firstly, allow me to clarify a few points. As you pointed out, spot instances are not meant to be run 24/7 - they are intended to provide extra compute capacity for a short time-span at a lower cost. Essentially, they are intended for tasks that can be broken into small pieces - so a terminated instance will not have a significant impact on the overall task.

That said, I have previously run an instance using the spot request model - its usual uptime was about 3 months - and I would typically overbid by about 15x the average market price. While this used to be a reasonably cost effective approach, I found that as more people employed the same technique, the price volatility increased to the point that it was no longer advantageous over a reserved instance.

The average spot price tends to hover around the hourly cost of a medium utilization reserved instance. Doing the math based on the current spot prices vs heavy utilization instances, you get the following:

+-------------+-----------------------------+------------------------------+----------+
|             |           1 year            |            3 year            |   Spot   |
+-------------+-----------------------------+------------------------------+----------+
| Small       | $0.016/h + $195 = $0.0383/h | $0.013/h +$300 = $0.0244/h   | $0.027/h |
| Medium      | $0.032/h + $390 = $0.0765/h | $0.026/h+$600 = $0.0488/h    | $0.038/h |
| Large       | $0.064/h + $780 = $0.153/h  | $0.052/h + $1200 = $0.0977/h | $0.108/h |
| Extra Large | $0.128/h + $1560 = $0.306/h | $0.104/h + $2400 = $0.195/h  | $0.216/h |
+-------------+-----------------------------+------------------------------+----------+

It is evident that in many cases, the 3 year heavy utilization reserved instance works out to less than the current spot price (which is subject to volatility). From my experience, the actual average spot price tends to be at least 50% higher than the baseline market value, since significant spikes are not uncommon.

Now, to actually try and answer your question:

You are correct that spot instances cannot be stopped - in some ways that does agree with the entire premise behind spot instances. Traditionally, an instance will create the EBS volume(s) it is using from snapshots, but it is possible to script the process of attaching an EBS volume. I used to use this script to setup 2 EBS volumes in a RAID0 (I am pretty sure I modified it from some script I found, but I can't find the original at the moment):

#! /bin/sh 
# 
# /etc/init.d/mountec2vol 
# 
# chkconfig: 234 20 50 
# description: Assigns an EC2 EBS Volume to a device and mounts the device 
# 
# To add this as a service run: 
# /sbin/chkconfig --add mountec2vol 
# 
# VARS 
# 
VOL1="vol-xxxxxxxa" 
VOL2="vol-xxxxxxxb" 
DEV1="/dev/sdh1" 
DEV2="/dev/sdh2" 
MOUNT_POINT="/raid" 

export PS1="[\T] [\W]# "
export JAVA_HOME=/usr/java/jre1.6.0_16
export EC2_HOME=/etc/ec2/apitools
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=/root/.ec2/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem 
export EC2_CERT=/root/.ec2/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem

MAX_TRIES=10 

# start/stop functions for OS 

start() { 
touch /var/lock/subsys/mountec2vol
INSTANCE=`curl http://169.254.169.254/latest/meta-data/instance-id 2> /dev/null` 
CTR=0 
/bin/echo "Mounting Elastic Block Store Volumes." 
ec2-attach-volume $VOL1 -i $INSTANCE -d $DEV1 
while [ ! -e "$DEV1" ]; do 
/bin/sleep 1 
CTR=`expr $CTR + 1` 
if [ $CTR -eq $MAX_TRIES ] 
then 
/bin/echo "WARNING: Cannot attach volume $VOL1 to $DEV1 -- Giving up after $MAX_TRIES attempts" 
exit 1 
fi 
done 
ec2-attach-volume $VOL2 -i $INSTANCE -d $DEV2
while [ ! -e "$DEV2" ]; do 
/bin/sleep 1 
CTR=`expr $CTR + 1` 
if [ $CTR -eq $MAX_TRIES ] 
then 
/bin/echo "WARNING: Cannot attach volume $VOL2 to $DEV2 -- Giving up after $MAX_TRIES attempts" 
exit 1 
fi 
done 
depmod -a
modprobe raid0
mdadm --assemble --verbose /dev/md0 /dev/sdh1 /dev/sdh2
if [ ! -d $MOUNT_POINT ]; then 
mkdir $MOUNT_POINT 
fi 
/bin/mount /dev/md0 $MOUNT_POINT 
} 

stop() { 
/bin/echo "Unmounting Elastic Block Store Volumes." 
rm -f /var/lock/subsys/mountec2vol 
/bin/umount $MOUNT_POINT
mdadm -S /dev/md0
ec2-detach-volume $VOL1
ec2-detach-volume $VOL2
} 


case "$1" in 

start) 
start 
;; 

stop) 
stop 
;; 
restart) 
stop 
sleep 5 
start 
;; 
*) 
echo "Usage: $0 {start|stop|restart}" 
exit 1 

esac 

exit 0 

This was for CentOS, but I imagine it would be easy enough to adapt to most other Linuxes. Change the Java version and remove the RAID attributes (and change the mount point) and you will be good to go. This would then be setup as an init script to run when the instance starts. An alternate way of accomplishing this would be to pass the EBS volume IDs as instance user-data, which would allow for a lot more flexibility.

It should be noted that this, of course, will not work for the root EBS volume - this setup presumes a mostly fixed root volume, with data kept on a separate mount-point. You can always set the EBS root volume from ec2-request-spot-instances using the --block-device-mapping parameter.