Understanding random testing

There are two good reasons to do random testing.

The first is that you’re working with a homogenous or expected distribution, and the cost of letting something defective slip through is pretty low. For example, the chef doesn’t have to taste every single serving before it leaves the kitchen. Quality manufacturing is based on the efficient use of random testing to be sure that each batch is expected to be within tolerance. It works as long as the thing being tested isn’t itself widely variable. If it is, you’ll need to test every single unit.

The second is that you’re trying to send a message to alter people’s behavior. Drug testing or tax audits are examples of this. You can’t test everyone, but you make it clear that there’s a non-zero chance that someone who is outside the rules will get caught.

We do random testing all the time without realizing it. That’s probably a mistake–we should either test every unit (when the stakes are high or when outcomes are unpredictable) or we should trust our people and our systems enough to test very rarely.

It doesn’t make sense to set up a random speed trap. Either measure the speed of every car or measure the speed of none of them.

Sending a message through testing and draconian punishment might make good security theatre, but it’s a waste of time and trust.

Seth Godin

Seth Godin

Hi. I’m Seth Godin. I’m a teacher, and I do projects.

For more than thirty years, I’ve been trying to turn on lights, inspire people and teach them how to level up. This blog has been appearing daily for more than a decade. One day, if we meet, I hope you’ll share with me your favorite posts. Even better, I’d like to hear about how a book or course helped you interact with the world differently and make a difference.
Seth Godin

Latest posts by Seth Godin (see all)

No tags for this post.

Leave a comment: