Cognitive Bias and Regression to the Mean April 29, 2014Posted by Peter Varhol in Software development.
Tags: bias, regression to the mean, testing
We prefer to assign causality to events in our own lives, and in the world in general. If something positive happens, we tend to credit our intelligence, or dedication, or some other quality. If negative, we often blame others, or perhaps blame our own failings. Every day when the stock market closes, we read about how stocks have gone up or down for some perfectly understandable reason.
Bull. Most things in our lives and in the world don’t happen for a reason, or at least any reason we can readily identify. Our good fortune may be only peripherally related to our intelligence or abilities, and our bad fortune may simply arise from being in the wrong place at the wrong time.
Regression to the mean is simply one example of our need for causality, and how it results in bias. If we perform exceptionally well, we come to believe our own press releases, and behave as though we are high achievers. We might well be, but achievement is a slippery thing; it might disappear in a heartbeat.
Regression to the mean is a statistical concept. It simply notes that any time you get an exceptional result, it is unusual. Subsequent results are more likely to be closer to the average. It’s a concept often found in the natural sciences. For example, I am taller than either of my parents, so it is likely that my children (if I had any) would be shorter than me, since I am taller than many of my direct ancestors.
Applied to our lives, regression to the mean refers to the fact that what we do is a combination of skill and luck. We have little idea how much is skill, and how much luck. When we do exceptionally well at a task, we tend to attribute that to skill. When we do poorly, we often blame bad luck. Instead, exceptional performances are random (and rare) chance.
You can certainly argue that such a statistical concept doesn’t really apply to individual efforts, but I think the general principle holds. Sometimes we simply do better than other times, and it’s not clear that it reflects skill any more than (good or bad) luck.
Applied to software development and testing, regression to the mean gives us preconceived notions of the performance of the software based on who works on it. It makes us believe certain things about software based on the perceived superiority or inferiority of the team members based on our experiences.