EC2, huge difference between top/sysstat and CloudWatch (cpulimit involved)

When you stop an EBS boot instance you are giving up the physical hardware (or slice of it) that the server was running on and EC2 is free to start somebody else's instance there.

Your EBS boot volume (and other attached EBS volumes) are still preserved, though they aren't really tied to a physical or virtual server. They are just associated with an instance id that isn't running anywhere.

When you start the instance again, EC2 picks some hardware to run it on, ties in the EBS volume(s) and boots it up again from the root volume.

Things that change when you stop/start include:

1. New internal IP address (though could randomly be the same).

2. New external IP address (though could randomly be the same).

3. If an Elastic IP address was associated with the instance before it was stopped, then you'll need to re-associate it after the start.

4. Any contents on the instance's former ephemeral storage were wiped and you are given fresh ephemeral storage.

5. You can leave an instance stopped for as long as you like and not get charged for run time (though you do get charged at a much lower rate for the EBS volume storage). See the next point.

6. A fresh billing hour is started for the instance when you start it again. E.g., if you start a new instance and then stop/start it 3 times within the first 60 minutes, you'll get charged for 4 hours instead of 1.

7. There is a small chance that EC2 will not have available slots of the correct instance type to run your instance when you want to start it again. I've had this happen and temporarily switched to a different, available instance type to get it running again.

When you reboot, it's a simple reboot at the OS level and the instance stays running on the same hardware, with the same private and public IP addresses, keeps the same Elastic IP address (if associated), and keeps the same ephemeral storage without getting wiped. No new billing hour is started on a reboot and you do not give up the instance hardware.

While an instance is stopped, you can do some cool things before starting it again. Here's an article I wrote on changing the instance type of an instance while it's stopped:

Moving an EC2 Instance to a Larger Size
http://alestic.com/2011/02/ec2-change-type

Here's an article I wrote on how to change the size of an EBS boot disk of an instance while it's stopped:

Resizing the Root Disk on a Running EBS Boot EC2 Instance
http://alestic.com/2010/02/ec2-resize-running-ebs-root

Here's an article I wrote on how to examine the root disk of an instance (while it's stopped) when you can't connect to it while it's runnning:

Fixing Files on the Root EBS Volume of an EC2 Instance
http://alestic.com/2011/02/ec2-fix-ebs-root

Since the stop/start cycle has a good chance of moving your instance to new hardware, it's an easy way to replace your instance hardware if you suspect that the current platform might be going bad and causing problems. Here's an article I wrote about that:

A Simpler Way To Replace Instance Hardware on EC2
http://alestic.com/2011/02/ec2-move-hardware

"Windows (Amazon VPC)" is a reserved instance specifically for EC2 instances that will reside inside a VPC. If your instance is not inside a VPC, then just purchase a "Windows" reserved instance.

This distinction is to prevent users from purchasing a reserved instance for inside a VPC, then wanting to have it apply to an instance outside a VPC (or vice versa).