I have had some perspective on this in the last few months & I believe these items to watch will address all the concerns above:
1) The comment from @Ross on the original posting is the key. T2 instances, no matter what scale and no matter whether they are EC2 or RDS, will stop performing when their CPU credits run out as the peak CPU demands continue.
2) The failure mode of a CMS web server we have seen most often is shown exactly by this condition: the CloudWatch graph dives towards zero when the CPU percentage needed by httpd
processes exceeds the CPU percentage assigned to that instance type (see doc link below).
3) The quick solution for a T2 instance that has exhausted CPU credits is to shut down, upgrade the instance type, and start up the instance again, which takes about 3-4 minutes. The most vital description of the capacities of different instance types is here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/t2-instances.html
4) Any production web server on AWS must have an Elastic IP address assigned in advance for this reason: if not, and the instance is rescaled, the IP address will change, leaving the web server inaccessible far beyond what would otherwise only be 3-4 minutes of downtime.
5) The only way to acquire more CPU credits is to upgrade the machine type. The amount of credits each T2 instance size can hold is described in the doc link above: it is always equal to the CPU work that instance type would do in 24 hours.
6) The machine can be returned to its original scale during a bit of scheduled downtime (again, 3-4 minutes) after peak performance demands die down.
7) I/O activity hasn't caused any performance degradation for our web server in any peak periods so far. The amount of IOPS is determined strictly by EBS volume size. Both the exact meaning of IOPS, and that relationship, are described here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html
8) Neither of the Cloud Watch metrics Freeable Memory nor DB Connections were of any use anticipating or correcting performance problems in a web server intensive environment.
I wouldn't recommend using the RDS DNS domain in my application config ie
*.rds.amazonaws.com
Set up a private DNS zone in Route53 instead. This is only available within your VPC, so you don't need to register it publicly or anything like that.
Then, set up CNAME records for your DB instances, which point to the RDS domain. Set very low TTLs (< 30s).
Then, configure your app to use the CNAME records.
If you then need to re-direct traffic to a new RDS instance, just update the CNAME record.
Also, if budget permits, you may want to consider using Multi-AZ within RDS. You'll pay twice as much, but its very handy in both scheduled and unscheduled failover situations.
Best Answer
What is the instance type that you are using for your RDS? Have you read the FAQ here -- http://aws.amazon.com/rds/faqs/#19
If you are already using the largest instance type see here --
Hope this helps