How much detail is in a good UI regression test

manual-testingqatestinguser interface

We use a detailed step-by-step user-interface regression test for our commercial web application. It has a "backbone" test for the most used / most important parts of the system, with optional tests for specific areas of functionality. Using this plan has definitely helped us ensure high quality software.

But, having very specific tests can be counter-productive. The tester concentrates on following the test and will completely miss usability issues, or not notice fairly obvious problems such as the bottom part of a page that is missing.

By contrast, some of the best UI testing happens when building a demo of a new feature. I often do my own best testing by pretending to demonstrate the system to an imaginary prospect. Yet when I tell the testers, "Just demonstrate the system to yourself" they don't cover nearly as much functionality as they do with a detailed point-by-point test.

I'm repeatedly asked to provide more and more detail in the test plan so that a new untrained tester can test with it without asking any questions. Yet details seem to be counter-productive. How much detail do you put in a regression test to make it effective? How do you write tests that make the tester focus more on the system than on checking off items on the test?

Best Answer

Technically speaking, a regression test should not be looking for usability issues. It may catch deviations from the UI design as part of the test cases but usability issues are best determined by UI design specialists. UI regression tests are usually performed by QA (testers), as you know.

If the product is too big for a detailed UI design, the UI designer produce should produce guidelines that the development team should follow and the QA should check for. For example, "Each page should contain a header with A, B and C" or "When asking for an address, it should have these fields laid out in this way".

It may be better to supply the UI verification steps in a different way. For example, instead of giving steps, include screen shots of that the app should look like (which should catch the bottom portion of a page missing).

You may also want to consider an automated UI testing tool like Selenium. This can automate much of the repetitive testing aspect you are capturing in the steps and free you and your testers up to look for harder to reproduce or more critical issues.

Related Topic