I would like to ship test code alongside a product. Specifically, provide an option so that anyone with a copy of our program can hit a "self-test" button or pass –self-test on the command line and run through the complete suite of unit | integration tests.
I mostly want to do this to help debug problems discovered in the field, so when a bug report is incoming from an end user there's a chance of it being supported by "and also, these three tests failed on my machine". I'd like manual testers to be able to run the unit | integration tests as well.
However, the test of the team believes that test code is not production code and therefore shall not be shipped. I don't really get this argument, since most open source projects ship a test suite. It does seem to be unusual in closed software.
I would like supporting evidence or anecdotes for either side of the argument. I've taken my best guess at which stack exchange site would be most appropriate, but please let me know if this is out of place.
Best Answer
Sometimes test code contains snippets of code from third parties, both external and internal to your company. This happens as users file bugs; your tests (such as regression tests) then incorporate the code supplied by them to reproduce. Often, the licensing of such code snippets to reproduce bugs is unclear. So, you should be aware of intellectual property issues. You don't want to ship test code that accidentally reveals some trade secrets or intellectual property of a different department of your company, or of your external partners either.
On another note, test code is rarely held to production code standards: code reviews aren't necessarily done; coding standards not enforced, etc.. This is unfortunate, yet commonplace, and should not necessarily a reflect poorly on the test team if they didn't have that goal in place at the time these tests were developed.
On the other hand, many tests are simply embarrassingly bad, and don't even test what some think is being test. That is different issue...
Ultimately, because of all these factors, you might want to classify your tests as those that can be shipped
as open source, and those that simply can't. (You may want to write some custom tests with shipping them in mind, slowly migrating the others into that set.)