The public DNS name always matches the public IP address.
The public IP address stays the same for an instance until it is terminated or stopped. A reboot does not change the public IP address.
If an EC2 instance is in a VPC, then it will retain the same public IP address across a stop and start.
If an EC2 instance that is not in a VPC is stopped and then started started again, it will probably receive a different public IP address.
Instances can fail. When you start a new instance to replace a failed or terminated instance, it will probably receive a different public IP address.
Because instances can fail, and because you may want to change the size of an instance (with a stop/start) it is not recommended to "ship an app that's wired to [the public IP address]" (or DNS name). Once your instance is stopped/terminated/failed another user could get that IP address assigned to their instance and all your traffic would go to them.
It is recommended to use Elastic IP Addresses to associate public services with your instance. You get to keep the Elastic IP address and you can assign it to any instance you want over time, even if it's the same instance after a stop/start.
Each Elastic IP Address comes with a public DNS name, but you would probably be better off mapping your own hostname to the Elastic IP address so that the name makes more sense to humans.
Here's a guide to Elastic IP Addresses:
http://aws.amazon.com/articles/1346
Here's an article I wrote that talks about the differences between rebooting and stop/start of an instance:
Rebooting vs. Stop/Start of Amazon EC2 Instance
http://alestic.com/2011/09/ec2-reboot-stop-start
Here's an article I wrote that provides a reason you may want to stop/start an instance even though you don't think you will today:
Moving an EC2 Instance to a Larger Size
http://alestic.com/2011/02/ec2-change-type
Best Answer
The public DNS name (whether elastic IP address or not) is exactly the same as using the public IP address (elastic IP or not) with the one following important difference:
You can use this trick with or without Elastic IP addresses. I recommend using Elastic IP addresses as it keeps the public DNS name the same even after stop/start or moving your service to another EC2 instance. Because of this, you can always use the public DNS name of the Elastic IP address and it will resolve to the internal IP address of the current instance to which the Elastic IP is associated.
You can extend this by using a CNAME DNS entry where you map your preferred hostname to the external DNS name of the Elastic IP.
Here's an article I wrote about using this feature to save money and speed up network performance with internal EC2 communication without having to keep track of the current internal IP address for each instance on all your other instances: http://alestic.com/2009/06/ec2-elastic-ip-internal
Other than this one difference, I agree that you might as well use the public IP address instead of the public DNS name because:
You save time by not doing a DNS lookup
You avoid any security risks that occasionally arise in the DNS protocol.
so I suppose, in reality, right there are two more differences...