Dr. James Whittaker's keynote has certainly been the hot topic of conversation at StarWest. Since I may be the only person here who agrees with him, let me share my take.
First, lets be clear that he was talking about "testing" as in the kind of testing done by testers in a testing department. He is not saying that no one will ever again run software to evaluate some characteristic of it. Just that it won't be done by testers as they exist today doing testing as we understand it today.
Second, the improvements in quality that have been made over the past decade do not involved testers testing software. He mentioned several improvements. Some, like better programming practices include using testing in new ways (like TDD). Some, like conformance to standards, involve testing not at all. But, the improvement in quality has not been due to testers using new testing practices. I last attended a STAR in the early 90s and the testing practices we had back then are by-and-large the same ones we use today. Even if they have new fangled names.
Third, he contends that the act executing software to find bugs can be done more cheaply and effectively through crowd-sourcing than by dedicated testers. Yes, I'm sure he knows that not all software is on the web and not all software testing can be replaced by crowd sourcing. But, to echo a comment in a previous session, if you have a software product and you don't think that someone, somewhere is trying to build something on the web to replace it, than you aren't paying attention. (Those are not the embedded systems you are looking for....)
As a result, if your only value comes from running tests to find bugs, you are on the path to obsolescence. Both because people are finding ways to build software that works well enough without doing that kind of testing and because even those companies that will continue to use that use finding and fixing bugs as a development strategy will find cheaper ways to let users do it. Sooner or later they will come take your job away.
The only place I differ is in the rate of change. Googlers tend to see the world changing at the speed of light. I look around and see the continued reliance on obsolete management practices, obsolete software systems and languages, and obsolete development practices, and come to the conclusion that inertia is much more resistent in the face of change that we think. Of course, that doesn't mean that I want to be trapped there.
"I look around and see the continued reliance on obsolete management practices, obsolete software systems and languages, and obsolete development practices, and come to the conclusion that inertia is much more resistent in the face of change that we think."
ReplyDeleteI'd use different rhetotric, but I think we've got substantial agreement here.
I've done subcontract work for Google through uTest. So, basically, google /does/ employ testers -- it just views the work they provide as a commodity that can be outsourced. In that desire to get testing work, but get it cheap, google is no different than most big North American companies.
They want to get every (X), but get it cheap, except perhaps senior management ... mostly because it is the senior managers who are making that decision. :-)
Here's the thing: If you want to crowdsource your testing, if you want to use production monitoring instead of testing, then your company needs certain instructure in place --
- You'll want to spend a lot of $$ on monitoring
- You'll need the ability to rollback on demand - including multiple change sets
- You need to get a lot of eyeballs on the software with low cost of failure
- You'll want the ability to slowly expand the circle of people who have the software, from your team, to the company, to friends and family, to your county, your state, USA, the world
To me, this sounds a lot like IMVU, FlickR, etsy.com, Groupon, and that well-funded series-B round startup down the street.
The insurance company, the bank, the state of california payroll system, or any new technology without testability hooks? (Think the web in 2001)
Turns out, not so much.
The 'traditional' test market might indeed be shrinking. To borrow a line:
"It was easier in the old days, of course, and society had more rats when the rules were looser, just as old wooden buildings have more rats than concrete buildings. But there are rats in the building now as well. Now that society is all ferrocrete and stainless steel there are fewer gaps in the joints. It takes a very smart rat indeed to find these openings. Only a stainless steel rat can be at home in this environment."
There may be fewer testers tomorrow, and we may be stainless steel.
But going away? I don't think so.
http://www.stickyminds.com/sitewide.asp?function=DETAILSIDX&tvniu=1&sqry=*Z(SM)*J(COL)*R(createdate)*&ObjectId=15502&ObjectType=COL&sidx=115
Matthew, I really like your points. It gives me something to think about: TDD and unit testing change the role of testers. I think that's been happening for a while. I appreciate you pointing out where crowdsourcing works and what it takes. I don't think it works for every product or company or user.
ReplyDelete