Decisions, Decisions – There’s an Algorithm for That March 20, 2017Posted by Peter Varhol in Software development, Strategy, Technology and Culture.
Tags: Kahneman, statistics, technology
add a comment
I remember shoveling sh** against the tide. Yes, I taught statistics and decision analysis to university business majors for about 15 years. It wasn’t so much that they didn’t care as they didn’t want to know.
I had more than one student tell me that it was the job of a manager to make decisions, and numbers didn’t make any difference. Others said, “I make decisions the way they are supposed to be made, by my experience and intuition. That’s what I’m paid for.”
Well, maybe not too much longer. After a couple of decades of robots performing “pick-and-place” and other manufacturing processes, now machine learning is in the early stages of transforming management. It will help select job candidates, determine which employees are performing at a high level, and allocate resources between projects, among many other things.
So what’s a manager to do? Well, first, embrace the technology. Simply, you are not going to win if you fight it. It is inevitable.
Second, make a real effort to understand it. While computers and calculators were available, I always made my students “do it by hand” the first time around, so they could follow what the calculations were telling them. You need to know what you are turning your decisions over to.
Third, integrate it into your work processes. By using machine learning to complement your own abilities. Don’t ignore it, but don’t treat it as gospel either.
There are many philosophical questions at work here. Which is better, your experience or the numbers? Kahneman says they are about the same, which does not bode well for human decision-making. And the analysis of the numbers will only get better; can we say the same thing about human decision-making?
Of course, this has implications to the future of management. I’ll explore my thoughts there in a future post.
Weapons of Math Instruction February 15, 2017Posted by Peter Varhol in Education, Technology and Culture.
Tags: data, Math, statistics
add a comment
That old (and lame) joke, of course, refers to Al-Gebra (algebra). But the fear of math is very real. For decades, many have hid behind the matra “I’m not a math person”, without exploring the roots of that statement. This article, by Jenny Anderson on Quartz, offers hope that we may be able to move on from this false rhetoric.
I never understood math early, but I always loved it. Post-BA degree, I taught myself calculus, and obtained an MS in applied math.
I taught various math and statistics courses to college students for 15 years. I would like to think that my enthusiasm and down-to-earth explanations at the very least made it tolerable to them. I still remember one student saying to me, “In elementary school, the teacher would preface the math lesson by saying, ‘I don’t want to do this any more than you do, but we have to, so let’s get it over with.’” I think teaching is a very big part of the problem. If teachers don’t like the topic, neither will their students.
I especially came to appreciate word problems, something that few if any students liked. I had a method of dealing with them. My original issue with word problems was that if I read it once and didn’t immediately see the solution, I would be stumped. Instead, I taught people to read the problem first, to understand it without seeking a solution. Then read it again, and highlight any information that seemed pertinent. Then read it a third time, to pull out that information and see how it might help lead to a solution. Then try a formula. If it didn’t seem to work out, discard it and start back at step 1.
It is not hard, folks, though it does require overcoming age-old biases, as well as a willingness to be open to new ways of thinking. Anderson notes that learning and applying math and quantitative methods requires a growth mindset. That is, a willingness to get something wrong, and learn from it for the future.
As we move (or already have moved) into a data-driven world that requires an intimate understanding of how data shape our lives, we can no longer plead ignorance, or lack of ability. If we plead lack of interest, we will be left behind.
We Don’t Understand Our Numbers March 27, 2016Posted by Peter Varhol in Strategy, Technology and Culture.
add a comment
I recently bought The Cult of Statistical Significance: How the Standard Error Cost Us Jobs, Justice, and Lives, by Stephen T. Ziliak and Deidre N. McCloskey.
Here’s the gist. Statistics is a great tool for demonstrating that a difference found between two sampling results is “real”. What do I mean by real? It means that if I measured the entire population, rather than just took samples, I would know that the results would be different. Because I sample, I have uncertainty, and statistics provide a way to quantify the level of uncertainty.
How different? Well, that’s the rub. We make certain assumptions about what we are measuring (normal distribution, binomial distribution), and we attempt to measure how much the data in each group differ from one another, based on the size of our sample. If the two types of results are “different enough”, based on a combination of mean, variation, and distribution, we can claim that there is a statistically significant difference. In other words, it there a real difference in this measure between the two groups?
But is the difference important? That’s the question we continually fail to ask. The book Reclaiming Conversation talks about measurements not as a result, but as the beginning of a narrative. The numbers are meaningless outside of their context.
Often a statistically significant difference becomes unimportant in a practical sense. In drug studies, for example, the study may be large enough, and the variability low enough, to confirm an improvement with an experimental drug regimen, but from a practical sense, the improvement isn’t large enough to invest to develop.
My sister Karen, a data analyst for a medical center, has pointed out to me that significance can also be in the other direction. She collects data on patient satisfaction, and points out that even minor dissatisfaction can have a large effect across both the patient population and the hospital.
That’s just one reason why the measurement is the beginning of the conversation, rather than the conclusion. The number is not the fait accompli; rather, it is the point at which we know enough about the subject to begin talking intelligently.
I Am 95 Percent Confident June 9, 2013Posted by Peter Varhol in Education, Technology and Culture.
Tags: big data, statistics
add a comment
I spent the first six years of my higher education studying psychology, along with a smattering of biology and chemistry. While most people don’t think of psychology as a disciplined science, I found an affinity with the scientific method, and with the analysis and interpretation of research data. I was good enough at it so that I went from there to get a masters degree in applied math.
I didn’t practice statistics much after that, but I’ve always maintained an excellent understanding of just how to interpret statistical techniques and their results. And we get it wrong all the time. For example:
- Correlation does not mean causation, even when variables are intuitively related. There may be cause and effect, or it could be in reverse (the dependent variable actually causes the corresponding value of the independent variable, rather than visa versa). Or both variables may be caused by another, unknown and untested variable. Or the result may simply have occurred through random chance. Either way, a correlation doesn’t tell me anything about whether or not two (or more) variables are related in a real world sense.
- Related to that, the coefficient of determination (R-squared) does not “explain” anything in a human sense. There is no explanation in our thought patterns. Most statistics books will say that the square of the correlation coefficient explains that amount of variation in the relationship between the variables. We interpret “explains” in a causative sense. Wrong. It’s simply that the movement between two variables is a mathematical relationship with that amount of variation. When I describe this, I prefer using the term “accounts for”.
- Last, if I’m 95 percent confident there is a statistically significant difference between two results (a common cutoff for concluding that the difference is a “real” one), our minds tend to interpret that conclusion as “I’m really pretty sure about this.” Wrong again. It means that if I conducted the study 100 times, I would draw the same conclusion 95 times. And that means five times I will draw the opposite conclusion.
- Okay, one more, related to that last one. Statistically significant does not mean significant in a practical sense. I may conduct a drug study that indicates that a particular drug under development significantly improves our ability to recover from a certain type of cancer. Sounds impressive, doesn’t it? But the sample size and definition of recovery could be such that that the drug may only really save a couple of lives a year. Does it make sense to spend billions to continue development of the drug, especially if it might have undesirable side effects? Maybe not.
I could go on. Scientific experiments in the natural and social sciences are valuable, and they often incrementally advance the field in which they are conducted, even if they are set up, conducted, or interpreted incorrectly. That’s a good thing.
But even when scientists get the explanation of the results right, it is often presented to us incorrectly, or our minds draw an incorrect conclusion. A part of that is that a looser interpretation is often more newsworthy. Another part is that our minds often want to relate new information to our own circumstances. And we often don’t understand statistics well enough to draw informed conclusions.
Let us remember that Mark Twain described three types of mendacity – lies, damned lies, and statistics. Make no mistake, that last one is the most insidious. And we fall for it all the time.