This post is mainly for me to have a place to collect model accuracy stories. If you see ones I missed, please add them to the comments.
Well, Dr. Birx just said it. Anyone in U.S. who dies with Covid 19, regardless of what else may be wrong, is now being recorded as a Covid 19 death.
— Brit Hume (@brithume) April 7, 2020
Yesterday I showed the deaths/cases plot. I speculated the “plot is still rocketing northwards means (1) cases are under-reported (lack of measurement), (2) deaths over-attributed (dying with is not same as dying from), or (3) both.” (1) is surely happening, but Hume’s tweet is proof (2) is, too.
Adjusting numbers is one good way to boost numbers to get them to align closer to forecasts.
Blaze has a list (thanks to David Legates for this) of model predictions.
Summary (all are quotes):
- Think Global: Total U.S. deaths: 1.64 million. Social distancing taken into account? No.
- CDC: Total U.S. deaths: 200,000 to 1.7 million. Social distancing taken into account? No.
- Imperial College of London: Total U.S. deaths: 2.2 million. Social distancing taken into account? … might reduce peak healthcare demand by 2/3 and deaths by half.
- University of Massachusetts – Amherst: Total U.S. deaths: 195,000 by the end of the year. Social distancing taken into account? …had the option to account for measures in their answers.
- The Coronavirus Task Force Model: Total U.S. deaths: 100,000 to 240,000. Social distancing taken into account? Yes
- Institute for Health Metrics and Evaluation: Total U.S. deaths: 81,766. Social Distancing taken into account? Yes, model assumes full social distancing through May. [MAY!]
BMJ has one of the first papers on the subject: Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal, by Wynants et al.
…27 studies describing 31 prediction models were included…
This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones…Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions.
This is harsh considering we’re not even at the end, yet.
The AP has an article by the usual suspects expressing wonder scientific models can be wrong: Modeling coronavirus: Uncertainty is the only certainty’.
The only problem with this bit of relatively good news? It’s almost certainly wrong. All models are wrong. Some are just less wrong than others — and those are the ones that public health officials rely on.
Welcome to the grimace-and-bear-it world of modeling.
“The key thing is that you want to know what’s happening in the future,” said NASA top climate modeler Gavin Schmidt. “Absent a time machine you’re going to have to use a model.”
Schmidt is right only in the philosophical sense that if you have to make a decision about some thing, you have to base that decision on something, even if that something is a coin flip. That’s a poor model, but it is a model.
The model updated this week by the University of Washington — the one most often mentioned by U.S. health officials at White House briefings — predicts daily deaths in the U.S. will hit a peak in mid-April then decline through the summer.
Their latest projection shows that anywhere from 49,431 to 136,401 Americans will die in the first wave, which will last into the summer. That’s a huge range of 87,000. But only a few days earlier the same team had a range of nearly 138,000, with 177,866 as the top number of deaths. Officials credit social distancing.
Well of course they do, because that’s what they ordered. Nobody is crediting nature, which is what “cured” us of Asian flu (2 million dead worldwide), Hong Kong chop suey fluy (1 million), Swine flu (~300,000 dead), and year after year of flu-flu (max ~650,000 per year).
It is not proof that because death totals come in smaller than the best case with full social distancing that social distancing is working “even better” than expected. The lower totals are just as much evidence that normal things like herd immunity, the weather, and so on were not accounted for properly in the models.
Here is a fellow wanting to maintain anonymity who makes the same point:
Herein lies the dilemma, or Sophie’s choice, of dealing with COVID-19. A full quarantine will result in the deaths of more elderly and medically ill people because more of them will become infected. A partial quarantine would likely result in a greater number of mild infections in young and healthy individuals upfront (but not total).
How many more elderly or medically ill people will die due to a full quarantine? It is hard to say, but a conservative estimate would be 5-10 times the number of young and healthy people who may die from a partial quarantine, based on fatality rates published by the CDC.
Fortunately, I am not responsible for making policy.
The author is an academic physician and researcher at an Ivy League institution in New York City.
To support this site and its wholly independent host using credit card or PayPal (in any amount) click here