Recently, I drove down to the video rental store. All my favorite movies—Plan 9 from Outer Space, Glen or Glenda, Santa Claus Conquers the Martians, Troll 2, and Escape from L.A.—were checked out, but I found a movie I hadn’t seen before—A Day Without a Mexican—and thought I’d give it a look.

The premise is simple: One day, without warning, everyone of Latino descent in California, a third of the state’s population, mysteriously disappears. As a pink fog surrounds California cutting off both travel and Internet access, cooks, gardeners, nannies, farm and construction workers, entertainers, athletes, as well as the fastest growing market of consumers, vanish.

California is in shock. Experts pose questions and offer answers: A mass UFO abduction? Genetically targeted bio-terrorism? The Rapture and Latinos are the chosen ones? Or perhaps they just left because they were tired of being underpaid and taken for granted.

The film is not very good (although it did win the award for best editing at the Guadalajara International Film Festival). It can’t decide whether it wants to be a goofy comedy, biting satire, or thoughtful social commentary. Ella Taylor of the L.A. Weekly described the movie as “A terrific premise, mangled to a pulp, then beaten to death in this forced mockumentary.” E! Online was less kind, stating, “Day not only lacks Mexicans, but also good acting, sharp storytelling, and humor.” Marjorie Baumgarten of the Austin Chronicle wrote, “Its narrative concept will entertain for a while, but eventually you will long to disappear with the rest of the Mexicans.”

And now for something completely different. (Don’t worry; it’s like a Tom Clancy novel. This all comes together in the end.)

“Testing costs too much.” “Testing takes too long.” “Testing is a giant sinkhole for our organization’s resources.” Heard these expressions of frustration before? Sure. If you’re in the testing profession, you’ve heard them repeatedly. And, if you’re like most testers, you’ve quickly retorted, “Does not!” or some other equally well-reasoned, quantitatively based response. Our difficulty is that most organizations have never attempted to quantify the value of testing. The cost—all those people and tools—is simple to compute. But the value—the economic benefit that comes from testing, minus its cost—is more difficult.

Every time testing discovers a defect that is repaired before it escapes to the customer, we have saved money for our organization and, therefore, created value. Specifically, we have saved the cost of dealing with the defect report, re-creating the problem, tracing from the failure to the underlying defect, fixing the defect, testing the fix, testing the test that tests the fix, and redistributing the software. Quantifying the value of testing is simply a matter of identifying, collecting, and summarizing those costs. A simple clerical task. Well, actually, a complex clerical task. It seems like a lot of work, so most organizations don’t do it. I suggest we don’t attempt to do it.

Rather, I propose a grand experiment. Instead of doing the work to quantify the value of testing, just ship the next release of your product with no testing at all—a release without a tester. (I told you it would come together.) No reviews, no inspections, no unit tests, no integration tests, no system tests, no acceptance tests, no performance tests, no security tests, no usability tests, no tests of any kind. Then, sit back and watch what happens.

I predict a major catastrophe—systems that give incorrect results while appearing to run normally, systems that freeze and give no results at all, massive loss or corruption of data, and correct results presented in indecipherable, confusing, and unusable ways. The aftermath will be the disintegration of organizations, the collapse of economies, and the loss of lives. It will be a sight to behold—even better than the movie—a truly convincing display of the value of testing.

Of course, if this is too radical an approach, you can do that boring old “bean-counting” exercise of value calculation. But I keep hoping someone will try my experiment.

Files: 

A Release Without A Tester