A lot has been written about the accuracy of the models that, earlier this year, predicted staggering death tolls from Covid-19 in countries around the world.
Why were these models so wrong? Looking at all the data and everything that has happened, I think there are XX possible answers.
- Bad data
___STEADY_PAYWALL___
The coronavirus is a novel virus. By definition, it is one we have not encountered before. Therefore, we don’t know terribly much about it.
We know a bit about other coronaviruses, it’s true. But this one, the virus that causes Covid-19, is still quite mysterious. We don’t know why it causes neurological symptoms – including stroke, the previously rare childhood disease known as Guillain-Barre syndrome, and general damage to the nervous system. We don’t know why it causes heart damage and blood clotting disorders. We don’t know why Covid-19 patients show up with dangerously low blood-oxygen levels and rampant pneumonia yet hardly any breathlessness. We don’t even know, really, how it spreads or how fast it spreads.
Epidemiological models that are used to predict potential death rates need good information about how infectious a disease is and how deadly it is. To be truly accurate, the model needs to know the health status of the population in question – how many older people, how many obese people, how many cancer survivors are there in the total population? It also needs to know how dangerous the disease is to different sub-groups within the population. For example, we know that the flu is dangerous for infants and young children, so we can assume that a flu outbreak at a creche will be deadlier than an outbreak at an office building full of adults. Finally, it needs to know how fast the virus is likely to spread in a specific population.
We don’t really know any of this about the novel coronavirus. A lot of the early information that most models were based on came from China – specifically, from Wuhan. It may not shock you to learn that there is evidence that the Chinese government was not completely open and transparent about what went down in Wuhan (or, for that matter, Hubei province as a whole). Plus, even the information that they did gather and share was incomplete. In the early days of Covid-19, Chinese authorities only tested sick people with fevers. They didn’t catch the huge group of infected, asymptomatic people out there.
Plus, the people in question had their unique characteristics. Lots of Chinese men smoke, for example, more than in places like Western Europe or the United States.
So, early models were working with incomplete and somewhat unreliable data. They tried to adjust for this, of course. They tried to build in assumptions about how different populations would respond, or what things would look like if the disease turned out to be more or less infectious than expected.
But, inevitably, early models were based on limited data and guesswork. The scientists behind them said as much themselves. They were estimates, the best possible estimates under the circumstances, but estimates nonetheless. As we learn more, the models improve. But they can only be estimates because there is so much we don’t know.
- Interventions happened
A major reason why Covid-19 has not been as deadly as predicted is that we did stuff to avoid that. Many people around the world have spent the last three months washing their hands repeatedly and staying in their homes. Globally, travel – especially international travel, but also regional and local travel – has declined sharply. People stopped going out. They stopped moving around. Bars and restaurants closed.
All of this had an effect – it meant that the coronavirus had fewer opportunities to spread and therefore, led to fewer cases of Covid-19 than expected because the expectation was based on a situation where we did nothing to stop it. In other words, there have been fewer deaths not because the models were wrong, but because we took actions to prevent those deaths.
Think of it this way. Let’s say you’re a morbidly obese, pack-a-day smoker. Your doctor tells you that, because of your weight and your smoking, she expects you to be dead within five years. So, you quit smoking. You start exercising. You eat right. You lose weight. You get fit and healthy.
Now, does it make sense for you to sue your doctor when you don’t die five years later? No, obviously not. Her prediction was based on your situation at the time and the assumption it wouldn’t change. The situation changed and invalidated the prediction. That doesn’t make the prediction wrong. If you had an identical, morbidly obese, smoking twin who didn’t make the lifestyle changes you made, they would be dead as predicted.
It’s nonsensical to complain about a lack of Covid-19 deaths when we have taken extreme measures to avoid them. The models weren’t wrong – we have no way of knowing, actually, if they were wrong or right because we don’t have a second, identical planet earth where there were no lockdowns or public health interventions – we just changed the situation.
- The downside risk was bigger than the upside risk
The third reason why epidemiologists got things wrong was that this was a moment, if ever there were one, to err on the side of caution. Nobody wants to be the @#$$hole who predicted no deaths, told politicians not to do anything about Covid-19, and then had to watch a million body bags pile up.
This comes down to how we think about risk. We all regard a small risk of a bad outcome as much more significant, emotionally and mentally, than a larger risk of a good outcome. When human being sat down to create models of Covid-19, they wanted to be very careful. They knew that if their models predicted a small death toll and the actual death toll turned out to be very high, they would shoulder some of the blame for those deaths. And the stakes, in this case, are very high.
Let’s say I came up to you and offered you a bet. I would randomly pick a number out of a hat containing 100 numbered pieces of paper. If I picked number 26 or higher, I would give you R1m. If I picked 25 or lower, I would shoot you three times in the head. So, you have a large, 75% chance of a positive outcome and a small chance of a catastrophic outcome. Death is very final and irreversible. It’s not, say, you owing me R1m. It’s death.
Very, very few people would take that bet. And, if you think you would take it, you’d probably change your mind as I reached into the hat with a .44 in my other hand.
Epidemiologists made a similar choice. They knew that, if they chose overly optimistic assumptions about infectiousness or lethality, they could very literally be sentencing people – perhaps their own friends and families – to death. So, they were cautious. It’s good that they were cautious. In this case, it is objectively morally better to be conservative, to assume Covid-19 is on the high end of the danger curve and motivate action to control it. Being too optimistic would cost, literally, millions of lives.
- Models answered one question only
The final reason why the Covid-19 models haven’t done what we wanted them to is that they were designed simply to estimate the potential death toll of Covid-19 (assuming a given level of preventative measures – none, in the case of early models).
The models were not designed to estimate the cost of the containment measures. They weren’t designed to estimate lockdowns’ impact on non-Covid-19 medical conditions or the economy.  The models were intended to give policymakers some of the information they needed to make a choice about policy responses to Covid-19, but they weren’t supposed to be the only information considered. The models were offered as information about the cost of doing nothing. Policymakers chose to do something.
