One of the most intriguing items in the recently released Visual Studio 2010 Feature Pack 2 is the ability to create and edit UI tests in Silverlight.
Here is an example of a coded UI test.
I haven't had much time to use it yet, but for people who have, I am curious as to what your thoughts are. Is this something you have found to be particularly useful?
I would like to be able to automate a significant amount of regression testing that we are currently performing manually.
In your experience, has it made a major impact on the resources that you would normally have to dedicate to testing?
Thanks
Best Answer
This boondoggle of automated UI testing is great in the short-term if you're running a QA department. Your staff will look super busy while they constantly write and re-write automated UI scripts. You can add another line on a report to brag about how much test coverage you do.
Over time, your department becomes a very obvious bottleneck that undermines productivity. Developers can't pass your automated tests because QA hasn't updated them to work against the new UI. This makes developers hesitant to introduce any UI changes.
Meanwhile, your business (or your client) isn't making the production release deadlines. The schedule slips while QA is scrambling to update the automated UI scripts against the new UI.
So, at this point, some reflection is in order.
The sell of automated UI testing is that it somehow liberates your QA staff from the testing effort required to go through the entire UI of an application.
What you've actually done, however, is replaced the tedious steps of user testing with the tedious process of scripting out those steps to a machine.
That initial cost would only be worth it if the steps are invariant, or if your application is very simple. It doesn't take science to determine that most UI development is anything but invariant. If your application is simple, then the testing effort is minimal.
I've seen two different companies try and implement this, and it's always ended in rolling heads and lawsuits.