I see several questions related to testing a Magento website including "How to test Magento upgrade", which discusses manual testing.
Are there any tools for automated regression testing of my upgrade? An ideal tool would learn my products etc. from the database (or previous crawl) and make sure everything is functioning after the upgrade.
I'm sure nothing will replace manual testing for many of the extensions and for the theme/template. But a general sanity check of an automated regression test would be valueable.
I might even use the automated regression after installing any new extensions.
Best Answer
Magento Test Automation Framework
The Magento Test Automation Framework (TAF) is used for precisely this reason. It leverages PHPUnit and Selenium to perform unit and functional tests. You can write your own custom tests for it as well.
The only downside is that it is deprecated. It hasn't been updated since Magento 1.7. So any new features will not be available for coverage.
Magento was written before testing was the norm inside of PHP, so there are areas where the TAF doesn't have 100% code coverage.
Here is the Github Repository.
Here is the installation guide.
Here is the user guide.
Ecomdev_PHPUnit
If you wanted to write some of your own PHPUnit tests, this is one of the best modules for doing so.
Download here.
Watir / BlueTir
The ruby web driver that allows for functional testing. Luis Tineo created a hybrid called "BlueTir" that is Magento specific.
Read about Watir here.
Download Blutir here.