During this past year, we’ve learned a lot about uncertainty and risk. And, we have have faced individual and collective decisions based on percentages: Do I need to wear a mask? Do we need a stronger lockdown? Who gets the vaccine first? Despite good use of scientific knowledge in countries like Germany, we’ve seen that this knowledge neither makes decisions easier, nor makes them politically more palatable nor guarantees a perfect outcome. Have we gotten better at using data and probabilities to take decisions? It's not clear that we have. But, do we have an alternative?
This year, the public discussion has actively engaged with the data and the probabilities to a degree rarely seen on any other issue. Twelve months ago, few could have correctly defined what the R-number means. Today, every newspaper reports it, without accompanying definition. Hundreds of millions have bookmarked the Johns Hopkins site with its daily counts and histograms. Excellent risk calculators show how behaviors like mask wearing mitigate risk./1/ Governments can furthermore rely upon much more detailed data backed by scientific assessments. It’s their job to distill those recommendations into workable and acceptable policy. In short, we have likely never had a crisis where we were better able to decide on a factual basis.
Still, data an probabilistic thinking remains hard to do. The mathematics is easy. On a simple level, it's just a percentage. Rather, the difficulty stems from their abstract nature as mathematical modelling tools.
The abstraction of probabilities
Simply put, a probability quantifies certainty. With 0% or with 100% chance of rain, I know whether to take an umbrella. With any value in between, I have uncertainty. Maximum uncertainty falls at 50% (p = .5). Note, this is not risk, just uncertainty. To have risk we need an impact, i.e. loss, damage or pain, that would result, if the event occurred. This is basic stuff: multiply probability with impact, and I know my expected loss.
The trouble with such numbers is that they are arid and abstract. Knowing the probabilities of infection doesn’t make the decision to visit grandmother any easier. As a result, some dismiss the science as wrong or irrelevant, rather than recognizing the very human dilemma in taking decisions and the responsibility for them, even when they turn out to be wrong. /2/
In addition, the size of the impact plays a role. The above scenario has the consequence of grandmother’s premature death. So, even a miniscule chance of infection seems too much. In a business scenario, consider the following two risks to a project whose potential benefit is € 1,000,000 in increased profit (if they risks don't occur):
- Risk A: probability is 80% that an impact (loss) of € 500,000 will occur.
- Risk B: probability is 4% that an impact (loss) of €10,000,000 will occur.
Lastly, probabilities mostly matter in cases where we need to take a decision. With them, we can quantify the decision case (as above). But, purely rational decisions don’t exist. Psychology shows that all decisions involve some emotion./3/ Daniel Kahneman has demonstrated how biased our fast thinking heuristics are, and how we would do well to do more slow thinking, i.e. carefully and critically weighing the evidence./4/ Nevertheless, we got where we are as a species equipped with both systems. As a result, even if we bother to calculate or model the numbers, we have powerful psychological tendencies pulling us toward taking gut decisions. It’s basic part of the human condition.
The gulf between decision and result
With all of these factors at play, it’s not hard to understand why, despite the growth in public awareness of probabilities, they remain hard to use effectively. By their very nature as a mathematical modeling tool, probabilities are separated from the actual result in a single case by an unbridgeable gap. The probability predicts the number of successful cases over many trials. A model that says we’ll break even 75% of the time also says that we can suffer losses 25% of the time. But, it says nothing about any single case. If, in the end, we lose money, it doesn’t mean the probability was wrong. Rather, it means that this time through was one of the 25%, not the 75%. A gambler, who plays her cards accordingly, will see that it balances out over many hands of cards./5/
In the pandemic, we can only run the trial once. Different countries’ actions and results does allow some degree of multi-trial comparison, and many countries have tried to learn from others’ actions. But, the complexity and differences from place to place are so high, that clear conclusions are elusive. If we apply some measure, such as a modest lockdown, we fundamentally cannot know if things would have been better or worse with a stronger one.
Our projects face the same dilemma. Usually, we only run a project once, i.e. we see only one trial. If we could run it multiple times, the 75:25 split predicted by the model would emerge. That doesn’t mean that the models are wrong or useless; such is the nature of uncertainty and the unidirectionality of time. We can, however, learn from the result and update our probabilities based on the knowledge gained from the failure.
An important corollary: just as a probability can’t predict an individual case, we can’t judge our decisions by the result. Generally, people are psychologically trimmed to take credit for success but attribute failure to outside factors. We want to believe that a great result means we took a great decision. With a 75:25 model, we are justified in choosing the 75% case. What if we lose? Does our decision become less great? The bias in this way of thinking protects our self-esteem, but it impedes learning. We’d do better to be more humble as well as more forgiving.
The course of the past year has made us acutely aware of how knotty the issues are, even when we trust the science. Should we just abandon the use of probabilities in our decision making? Personally, I conclude with a decisive, “no”. We need more numeracy, and the more we practice it, the better we get at employing the tools. In these pages, Jan and I have recommended Douglas Hubbard’s book, How to Measure Anything/6/, many times as a thorough and accessible introduction to using probabilities in decision making.
In the end, we need to ensure that our decision-making processes are robust. They need to use facts and probability models. We need to be honest and transparent, avoid HIPPO decisions and challenge our gut impulses with empirical measurements./7/ If a model doesn’t predict good results for a pet project, it should be scrapped. We need feedback loops to enable learning both from success and failure. A great decision is one that makes use of such resources—regardless of the result. A poor decision is one that does not—again, regardless of result. Fortunately, there is a general correlation between a good decision making process and good results, at least long term over many trials.
Notes
/1/ For example: https://www.zeit.de/wissen/gesundheit/2020-11/coronavirus-aerosole-ansteckungsgefahr-infektion-hotspot-innenraeume
/2/ The public health perspective, where statistics can model whether we’ll have enough hospital beds, is fundamentally different than the personal one. The decisions are also different, because the consequences are different. A country must generally protect it’s citizens, but cannot try to ensure maximal protection for every individual.
/3/ Jonah Lehrer, The Decisive Moment : How the Brain Makes up Its Mind (Edinburgh: Canongate, 2009). Among other evidence, Lehrer cites a study of persons whose left and right brain hemispheres are separated (from birth, accident or surgery), thus separating their rational from emotional capacities. Such persons can be highly adept at solving, for example, a math problem. After listing all of the pro and cons, they are nevertheless unable to choose which color pen for writing a letter.
/4/Daniel Kahneman, Thinking, Fast and Slow (Farrar Straus & Giroux, 2011).
/5/ Annie Duke, Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts (New York: Portfolio/Penguin, 2018).
/6/ Douglas W. Hubbard, How to Measure Anything: Finding the Value of Intangibles in Business, Third edition (Hoboken, New Jersey: John Wiley & Sons, Inc, 2014).
/7/ HIPPO = highest paid person’s opinion.
Kommentare
Kommentar veröffentlichen