Friday, November 4, 2011

On Testing Will Not Solve Your Quality Problems

I remember a conversation that I had with my Linear Algebra professor when I was in college about his class.  He said it was a classic example of a double humped class.  For half the class, he could teach the material twice as fast and it wouldn't be a problem.  For the other half of the class, however, he could spend twice the amount of time and they still wouldn't succeed.

In economics, they refer to this as multiple equilibria.  I believe that software quality has two equilibria.  We have understood for a decades now the types of practices that are needed to develop working software.  Whether it takes the form of code reviews, pair programming, or a gatekeeping committer, every line of code needs to be seen by multiple pairs of eyes.  Whether by convention or test driven means, you need need comprehensive unit tests.  Since code evolves, you need automated tests that cover the code to tell you when changes have had unexpected effects.

Creating working software requires the disciplined and diligent use of these techniques.  It requires both intention and effort.  It is, as a result, subject to Broken Windows effects.  When you start to loosen your practices, you send the message that quality is not a goal and create an environment where the number of quality problems begins to escalate.

When managers try to solve quality problems by doing more testing, they are really trying to avoid the cost of reaching the "working software" equilibria.  Unfortunately, the same lack of attention to quality that created the problems, sabotages our efforts to fix them as well.  Trying to take the cheap way out reinforces the social norm that quality is not really that important for developers.  We are simply unable realize a specific level of quality by dialing up or down our quality practices.  And testing alone will not enable us to reach the working software equilibria.

Testing will not solve your quality problems.



No comments:

Post a Comment