The satirist Ambrose Bierce once defined prophecy as the “art and practice of selling one’s credibility for future delivery.” Covid-19 has produced no shortage of doomsaying prophets whose prognostications completely failed at future delivery, and yet in the eyes of the scientific community their credibility remains peculiarly intact.
No greater example exists than the epidemiology modeling team at Imperial College-London (ICL), led by the physicist Neil Ferguson. As I’ve documented at length, the ICL modelers played a direct and primary role in selling the concept of lockdowns to the world. The governments of the United States and United Kingdom explicitly credited Ferguson’s forecasts on March 16, 2020 with the decision to embrace the once-unthinkable response of ordering their populations to shelter in place.
Ferguson openly boasted of his team’s role in these decisions in a December 2020 interview, and continues to implausibly claim credit for saving millions of lives despite the deficit of empirical evidence that his policies delivered on their promises. Quite the opposite – the worst outcomes in terms of Covid deaths per capita are almost entirely in countries that leaned heavily on lockdowns and related nonpharmaceutical interventions (NPIs) in their unsuccessful bid to turn the pandemic’s tide.
Assessed looking backward from the one-year mark, ICL’s modeling exercises performed disastrously. They not only failed to accurately forecast the course of the pandemic in the US and UK – they also failed to anticipate Covid-19’s course in almost every country in the world, irrespective of the policy responses taken.
Time and time again, the Ferguson team’s models dramatically overstated the death toll of the disease, posting the worst performance record of any major epidemiology model. After a year, some of the ICL predictions reach farcical territory. Their forecast of 179,000 deaths in Taiwan, which never locked down, was off by 1,798,000% (as of this writing, Taiwan has just 12 Covid-19 deaths). A similar story played out in other countries that eschewed the lockdown approach for the first year of the pandemic. Imperial overstated the predicted mortality of Sweden (392%), South Korea (17,461%), and Japan (11,670%) in the absence of heavier-handed NPIs than any of these countries actually imposed.
But what about the rest of the world? Most other countries experimented with some form of Neil Ferguson’s prescriptive advice over the last year, although for different degrees of severity and duration. Despite widely different mortality outcomes of their own, no other country provides anything approaching a clear validation of the ICL model.
The searchable results above (view full screen for best results), compared to the actual death toll on March 26, 2021 – one year after the original release of Imperial’s international model.
The table depicts three modeled scenarios that were published in ICL’s report from one year ago (ICL also included a fourth scenario attempting to approximate focused protection of elderly populations; however this approach was not meaningfully attempted in any country).
The first scenario shows an extreme “suppression” model, triggered when a country reached 1.6 deaths per 100,000 residents. This strategy envisioned a stunning 75% overall “uniform reduction in contact rates” across the entire population. Even in the short term, this approach is akin to the harsh measures first implemented in the Wuhan region of China as distinct from the lesser lockdowns with “essential business” exemptions seen in most of the world. But ICL’s suppression strategy also assumed that this measure “will need to be maintained in some manner until vaccines or effective treatments become available” – basically a full year or more of uninterrupted lockdown.
No country on earth maintained a 75% suppression rate of all contacts for an entire year, making ICL’s first model an extreme hypothetical of what a “best case” aggressive policy response could attain rather than a predictive reflection of reality. Despite its hypothetical nature, ICL’s suppression model still managed to overstate the number of Covid-19 deaths in all but the 20 worst-afflicted countries – none of which used anything close to the scenario’s policy approach.
The second ICL strategy is closer to reality in most countries. This “mitigation” model envisioned mandatory population-wide social distancing with a primary aim of preserving hospital capacity to treat the disease – a “flattening of the curve” as the popular slogan maintained. Using the most conservative replication rate that they modeled, R=2.4, Imperial’s “mitigation” forecasts managed to dramatically overstate the number of deaths in every single country on earth. Using a higher R0 yields even more extreme overpredictions. But sticking with the 2.4 scenario is sufficient to show the systemic problem in the ICL model. Their “mitigation” numbers were too high by roughly 20-30% in hard-hit locations such as Peru, Mexico, and the Czech Republic – all countries that used stringent lockdown measures at several points in the last year. On the other extreme, ICL overstated the “mitigation” scenario’s predicted death toll by 100,000% or more in a dozen countries. All but about 20 of the hardest-hit countries had “mitigation” forecasts that ran high by 100% or more.
The third ICL strategy projected the results of an “unmitigated” pandemic in which governments did nothing at all. This is the scenario that famously predicted 2.2 million deaths in the United States, 500,000 in the United Kingdom, and similar catastrophic outcomes across the world. Although Ferguson’s team has a bad habit of falsely claiming credit for saving millions of lives premised upon these apocalyptic numbers, the truth is they all amounted to wild exaggerations from a fundamentally flawed model. At the 1-year mark, no country on earth approached anywhere near ICL’s “unmitigated” projections, and certainly not any of the countries that avoided heavy-handed lockdowns.
Although ICL did not release its full timeline of how the pandemic would play out under these scenarios, its modeling enterprise was built upon the assumption that the peak daily death toll for each country would hit approximately three months after the introduction of the virus. For most countries, that means a predicted peak sometime in the summer of 2020, with the overwhelming majority of forecast deaths to have occurred by the end of that wave. A year later, most countries have not even remotely resembled the tolls predicted under most of the ICL model scenarios.
Several questions remain.
Why is Ferguson, who has a long history of absurdly exaggerated modeling predictions, still viewed as a leading authority on pandemic forecasting? And why is the ICL team still advising governments around the world on how to deal with Covid-19 through its flawed modeling approach? In March 2020 ICL sold its credibility for future delivery. That future has arrived, and the results are not pretty.
0 Comments