UAT Testing – Why UAT is Considered a Validation Activity

acceptance-testingterminologytestingvalidationverification

Virtually any book dealing with software testing mentions that user acceptance testing (UAT) is an ultimate validation activity, often quoting Boehms informal definition: "Validation: Are building the right product?"

But how come that Boehm himself considers UAT a verification?

V Model Diagram

GUIDELINES FOR VERIFYING AND VALIDATING SOFTWARE REQUIREMENTS AND DESIGN SPECIFICATIONS.
B.Boehm:

From the V-chart, it is clear that the key artifact idstinguishes
verification activities from validation activities is the software
requirements baseline. This refers to the requirements specification
which is developed and validated during the Plans and Requirements
Phase, accepted by the customer and developer at the Plans and
Requirements Review as the basis for the software development
contract, and formally change-controlled thereafter.

By definition,verification involves the comparison between the requirements baseline
and the successive refinements descending from it — the product
design, detailed design, code, data base, and documentation — in order
to keep these refinements consistent with the requirements baseline.
Thus, verification activities begins in the Product Design Phase and
conclude with the Acceptance Test
. They do not lead changes in the
requirements baseline; only to changes in refinements descending from
it.

On the other hand, validation identifies problems which must be
resolved by a change of the requirements specification. Thus, there
are validation activities which occur throughout the software
life-cycle, including the development phase. For example, a simulation
of the product design may establish not only that the design cannot
meet the baseline performance requirements (verification), but also
that the performance requirements are too stringent for any
cost-effective product designs, and therefore need to be changed
(validations).

The only explanation I can think of would be that Boehm did not consider Acceptance test as a USER acceptance test but as some kind of final check against the specification.

In addition, I believe this original article every book refers to is much clearer than those artificial definitions of V&V (that are, in addition, wrong from this point of view). According to Boehm, when I check the products against the requirements, it is a verification. When I test the product for its use, I validate it as I might find something that leads to changes in the requirements.

Of course Boehm's original idea contradicts some standards, such as ISO 12207, in which validation is to confirm that the requirements for a specific intended use of the software work product are fulfilled. Which means checking against the spec.

Most books says that verification is only static and the code is not run. I believe that according to Boehm, it is not true. E.g.:

Software Engineering And Quality Assurance, A.A.Puntambekar: The verification activities fall into the category of static testing.

Effective Methods for Software Testing: Includes Complete
Guidelines,William E. Perr: Verification testing – testing in a static
mode – ….. Validation testing (e.g. unit, integration …)

I would conclude that despite many authors' opinions, I believe VERIFICATION includes testing which is done in order to check the correspondence of the specification and the program. Validation includes testing as well but it would not be based on the specifications.

Best Answer

In the opening sections of the paper that you quote, Boehm states:

There is very little agreement in the software field on the definitions of the terms "verification" and "validation."

Boehm defines the terms verification and validation as he intends to use them in the rest of the paper, in order to be able to clearly present the rest of the writing without ambiguity:

Verification - to establish the truth of the correspondence between a software product and its specification. (Note: This definition is derived from the Latin word for "truth," veritas. Note also that data bases and documentation are fit subjects for verification, as well as programs.)

Validation - to establish the fitness or worth of a software product for its operational mission. (Note: This definition is derived from the Latin word for "to be worthy," valere.)

I think that the Guide to the Software Engineering Body of Knowledge can provide some insight into this as well:

Classical treatments suggest that “verification” deals with static evaluation methods and that “testing” deals with dynamic evaluation methods. Recent treatments, including ISO/IEC draft 29119, are blurring this distinction, though, so testing standards are mentioned here.

What all of this says is that, at the time Boehm wrote his paper, there wasn't a widely accepted singular definition for what is "verification" and what is "validation". Even today, that's still true.


Now that we know that there is some ambiguity in the terms "verification" and "validation" (and how "testing" fits into these), we can look at the diagram that Boehm presented.

Something that Boehm didn't discuss, but I've seen in multiple sources (including ISO/IEC 12207), validation specifically requires that the validation activities occur in the operational environment and can achieve its intended purposes, from the user's perspective. This may be implied by Boehm's definitions, but it's not clearly stated.

Boehm also used the term "acceptance testing". You are interpreting this as "user acceptance testing". There are other kinds of acceptance testing, such as factory acceptance testing (FAT). In my experience, a FAT is performed by the developing organization at the developing organization's site, with the procedures reviewed and accepted by the customer. This is a check to ensure the complete system (all software and hardware) are fully functioning, meet the specification, and a final inspection prior to delivery to the customer's facility. Based on Boehm's definitions, this can be a verification activity the developing organization can not fully exercise the product within the operational context.

What you are considering "user acceptance testing" is likely "OT&E" - Operational Test and Evaluation. This is indeed a validation activity since the customer and user are able to see the product within the operational context and say if it does or does not meet their needs.

Related Topic