Tuesday, October 25, 2011

On "Why didn't you find this bug?"

The question "Why didn't you find this bug?" is an organizational smell.  It reflects fundamental problems with way your organization thinks about testing and quality.

First, as Dawn Haynes points out, this isn't a "you" question, it is a "we" question.  The "you" divides the team.  A team that is functioning well, has collective ownership for delivering code that works and for the techniques used to remove bugs.  When you ask why "We didn't find this bug?", you open up the range of possible answers.

The question is usually asked in the context of "Why didn't you (tester) find this bug (when doing the feature/system/acceptance testing of the software)?  Notice the embedded assumption that regardless of how the software is written, by testing at the end, we should be able to remove all bugs.  If you expect to remove all problems by testing, you will be sadly mistaken.  Even when the test you planned should have caught the bug, the execution often does not go as planned because the software was delivered late or wasn't really ready to test.  Its a classic case of an unrealistic expectation that we usually undermine anyway.

But the real problem is that the question"Why didn't you find this bug?" is usually asked to avoid having to answer the question that really matters, "Why did we write the bug in the first place?" The answer to this question requires reflection and creates the responsibility to learn which is, perhaps, why organizations would choose to avoid it.  Better to imply blame and make you-know-who get better.

Until we really understand how we created the bug in the first place, we can't answer the question that really matters, "What is the cheapest thing we can do to prevent us from delivering software with this bug again?"  Since testing is one of the most expensive and least reliable of the techniques we use to remove bugs, the answer often lies elsewhere.  If you don't take steps to prevent the bug from being written in the future, you are giving permission to make it again.  And since we shouldn't count on testing to find it in all its future occurrences, we are giving permission to deliver it again as well.

No comments:

Post a Comment