On Why Are We Testing Anyway

The paradox of testing is, if testing has such limited value why do companies that produce great software do so much of it?  If testing can't really improve quality and doesn't really provide data to decision makers, why do we bother? What is the value of testing?

In my experience, testing is best understood as a finishing activity, a polishing activity.  It is the last few coats of clear paint that make a paint job shine.  It is running your hand across the surface of a finished product to find any remaining small burrs that need to be sanded over.  It is the attention to detail that differentiates value from junk.

This understanding of the purpose of testing unlocks the current trends and debates in the test community.  It explains why the traditional practice of testing as a planned, designed, and scripted activity is being replaced by rapid testing techniques.  If you have lots of the kinds of bugs that are uncovered by traditional "systemic" testing techniques, then testing can do nothing for you.  But, if you have done your development well, the kind of intelligent investigation that characterizes exploratory testing can uncover the problems you have missed.

Is testing dead?  If your business model does not require polished products, it may be.  When Alberto Savoia or James Whittaker talk about how testing can be replaced by pretotyping, dogfooding, or beta testing, they are really just saying that Google's business model doesn't require polished products.  For Google, it is more important to get customer feedback on the software's functionality than it is to get the software just right.  And that may be true for some software on the web just as it is surely not for software that, for example, is embedded in a device.

This does suggest that as companies begin to understand the real value of testing, they will have fewer but better testers. And this explains why outsourcing testing has not produced the expected benefits.  If you are trying to replace an intelligent exploratory tester with a phalanx of testers for whom testing is the script and nothing but the script, you get little value from it.

Testing, at least the kind of system level testing done by testers after development is "done," is part of the zen in the art of software development.  It is the attention to detail that differentiates software that users love from software that merely meets needs.