Zero defects

There are several expressions I have really come to hate over the years. Two of them – the concept of “zero defects” and the idea that “if it ain’t broke, don’t fix it” – are, fortunately, falling out of favor at long, long last.

“Zero defects” sounded good in the advertising, but became recognizable as a pretty dumb idea when accountants began calculating the costs involved in making sure that every last product that went out the doors was perfect. The cost of going from 99% perfect to 100% perfect (zero defects) was often astronomical however, and it became obvious that customers and vendors alike got a much better deal if vendors could hold down costs by allowing a small number of defective items to get out into the marketplace.

“If it ain’t broke, don’t fix it” has always been a pet peeve of mine. I view this sort of attitude as wholly appropriate if you are baking bread or mixing cement. In any high-tech business however, where things change at least as frequently as a hardware engineer changes his shirt (let’s say, weekly), this is a one-way ticket to Palookaville. Why? Because while your systems continue to operate effectively in their “unfixed” state, your competitors have improved their systems so that they now operate optimally. You have fallen behind the curve. The difference between effective performance and optimal performance often makes you a less effective competitor.

A third concept that likely belongs in the same bucket with the other two is “best-in-class.”

What does “best in class” mean, anyway? In some circumstances, “best” may be provable: quantitative analysis can show us what is fastest or most reliable, for example. If that were all there was to “best” though we could just accept modeled results for I/O and such and would never need actually to test something in our own environments. I am willing to bet that few of my readers are willing to take that chance however.

Much of the reason we don’t trust modeled results (and in many cases, vendor benchmarks) is the concern that the models we see – the ones used to prove “best-in-class” performance – don’t accurately reflect the environments into which we will be dropping the products. In other words, we understand the concept of “your mileage may vary” when the product shows up on our own shop floor. The reason for this variance is that the models represent pre-tested, clearly bounded test environments, and our sites do not.

I wonder what the impact would be if we were to nudge those boundaries a bit and create a more complex (and real) situation, but I don’t recall ever having seen vendor data indicating what happens when two such “best-in-class” systems are left running side by side. It would be useful to understand the effect one would have on the other. I am willing to bet that as we keep adding one such “best” system to another, increasing the complexity of the system, the overall environment becomes decidedly less deterministic.

Is the concept of best-in-class, when applied to the real world, a fraud? Maybe so.

If you have feelings about the idea of best-in-class – whether you believe in it or consider it nonsense – please drop me an e-mail. If you write it, I promise to read it.