Statistics

More On Coronavirus Model Accuracy

This post is mainly for me to have a place to collect model accuracy stories. If you see ones I missed, please add them to the comments.

Yesterday I showed the deaths/cases plot. I speculated the “plot is still rocketing northwards means (1) cases are under-reported (lack of measurement), (2) deaths over-attributed (dying with is not same as dying from), or (3) both.” (1) is surely happening, but Hume’s tweet is proof (2) is, too.

Adjusting numbers is one good way to boost numbers to get them to align closer to forecasts.

Blaze has a list (thanks to David Legates for this) of model predictions.

Summary (all are quotes):

  • Think Global: Total U.S. deaths: 1.64 million. Social distancing taken into account? No.
  • CDC: Total U.S. deaths: 200,000 to 1.7 million. Social distancing taken into account? No.
  • Imperial College of London: Total U.S. deaths: 2.2 million. Social distancing taken into account? … might reduce peak healthcare demand by 2/3 and deaths by half.
  • University of Massachusetts – Amherst: Total U.S. deaths: 195,000 by the end of the year. Social distancing taken into account? …had the option to account for measures in their answers.
  • The Coronavirus Task Force Model: Total U.S. deaths: 100,000 to 240,000. Social distancing taken into account? Yes
  • Institute for Health Metrics and Evaluation: Total U.S. deaths: 81,766. Social Distancing taken into account? Yes, model assumes full social distancing through May. [MAY!]

BMJ has one of the first papers on the subject: Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal, by Wynants et al.

…27 studies describing 31 prediction models were included…

This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones…Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions.

This is harsh considering we’re not even at the end, yet.

The AP has an article by the usual suspects expressing wonder scientific models can be wrong: Modeling coronavirus: Uncertainty is the only certainty’.

The only problem with this bit of relatively good news? It’s almost certainly wrong. All models are wrong. Some are just less wrong than others — and those are the ones that public health officials rely on.

Welcome to the grimace-and-bear-it world of modeling.

“The key thing is that you want to know what’s happening in the future,” said NASA top climate modeler Gavin Schmidt. “Absent a time machine you’re going to have to use a model.”

Schmidt is right only in the philosophical sense that if you have to make a decision about some thing, you have to base that decision on something, even if that something is a coin flip. That’s a poor model, but it is a model.

The model updated this week by the University of Washington — the one most often mentioned by U.S. health officials at White House briefings — predicts daily deaths in the U.S. will hit a peak in mid-April then decline through the summer.

Their latest projection shows that anywhere from 49,431 to 136,401 Americans will die in the first wave, which will last into the summer. That’s a huge range of 87,000. But only a few days earlier the same team had a range of nearly 138,000, with 177,866 as the top number of deaths. Officials credit social distancing.

Well of course they do, because that’s what they ordered. Nobody is crediting nature, which is what “cured” us of Asian flu (2 million dead worldwide), Hong Kong chop suey fluy (1 million), Swine flu (~300,000 dead), and year after year of flu-flu (max ~650,000 per year).

It is not proof that because death totals come in smaller than the best case with full social distancing that social distancing is working “even better” than expected. The lower totals are just as much evidence that normal things like herd immunity, the weather, and so on were not accounted for properly in the models.

Here is a fellow wanting to maintain anonymity who makes the same point:

Herein lies the dilemma, or Sophie’s choice, of dealing with COVID-19. A full quarantine will result in the deaths of more elderly and medically ill people because more of them will become infected. A partial quarantine would likely result in a greater number of mild infections in young and healthy individuals upfront (but not total).

How many more elderly or medically ill people will die due to a full quarantine? It is hard to say, but a conservative estimate would be 5-10 times the number of young and healthy people who may die from a partial quarantine, based on fatality rates published by the CDC.

Fortunately, I am not responsible for making policy.

The author is an academic physician and researcher at an Ivy League institution in New York City.

To support this site and its wholly independent host using credit card or PayPal (in any amount) click here

Categories: Statistics

31 replies »

  1. Dear Dr Briggs,

    you cite our recent study (Wynants et al.) that appeared in the BMJ yesterday. Your post is apparently about modeling the Corona virus, while our paper is not. Our paper is about the modeling of *individuals* in terms of their diagnosis or prognosis, not about the Corona virus itself. There is a big difference between these clinical prediction models and models of disease spread.

  2. Yes, I noted that the CDC is making up numbers and commented that it was happening.

    ALL models on Covid 19 are biased. IT’S NOT A VIRUS, IT’S A POLITICAL MOVEMENT. When you are pushing a political movement, lying is a requirement. SARS-CoV-2 is virus. We didn’t model it, except in very scholarly papers of researchers nearing retirement.

    “Based on National Center for Health Statistics (NCHS) mortality surveillance data available on March 26, 2020, 8.2% of the deaths occurring during the week ending March 21, 2020 (week 12) were due to P&I. This percentage is above the epidemic threshold of 7.2% for week 12.” CDC
    We are 1 percent over epidemic threshold, but no one cares. 24,000 people are estimated to have died to date (flu estimates really are no better than Covid, having huge uncertainty). Again, no one cares. No one models flu. MOST cases are H1N1–remember that? The pandemic no one cared in the least about????? That drained resources that were never replaced? The Obama gift that keeps on giving??

    This is insanity. Not a virus, not a disease in any sense. It’s evidence of utter brain death. There is no other way to describe it. And you cannot model insanity. You can keep trying, but you fail every time.

    Psychics are the correct way to know the future, not models. Use psychics.

  3. Let’s also consider elderly locked in nursing home or hospital wards who, instead of spending their last days with their families, can only interact with them via video. Is this even humane?

  4. I am Lord Fauci Thy God, thou shalt not have any strange gods before Me.

    Thou shalt not take the name of Lord Fauci Thy God in vain.

    Remember to keep holy the daily White House Briefing

    Honor Thy Father Fauci and Thy Mother Brix

    Thou shalt not turn off the TV

    Thou shalt not commit adultery by intellectually fornicating with other epidemiological models

    Thou shalt not steal thyself against Thy Lord Fauci’s orders

    Thou shalt not bear false witness against Thy Lord Fauci.

    Thou shalt not covet thy neighbor’s epidemiological models

    Thou shalt not covet thy neighbor’s masculine ideas about liberty.

  5. 2019-2020 Flu vs Corona in Virginia, as of 2 April 2020:

    Virginia Department of Health (VDH):

    Virginia deaths from pneumonia and influenza during the 2019-20 flu season: 1,352

    Deaths of children from influenza during the 2019-20 flu season: 5

    http://www.vdh.virginia.gov/…/Weekly-Influenza-Activity…

    Virginia deaths from C19: 63
    Virginia deaths of children from C19: 0

  6. jim fedeko: “Is this even humane?” Absolutely not. It’s cruel and evil. If one is near death and one does not want the nursing home or hospital, arrange to send them home. It’s far more humane.

    Even if you’re not in a nursing home, this is flat out cruel. My 93 year old mother in law is stuck at home. She can sit on her screen porch, check her flower gardens, but she cannot visit any of her children, grandchildren or great-grandchildren nor can she attend church or any social functions. She had a fuller social life than I ever would have considered. We’d call and she’d be out visiting somewhere. This is just cruel, cruel, cruel. We call her a lot and are sending flowers today, but it can only do so much. (Not all moms care, of course. Mine rarely left home. But to force people to give up their lives…..) I’m not sure if her grandkids could sing to her on the sidewalk or any such thing–not sure if the police state is in full force where she lives. It’s just so sad, so not necessary. Time to rebel.

  7. The official prognostication is now 60,000 US deaths from the Chinese Designer Virus. Furthermore, as many as 98% of those have comorbidities: heart disease, diabetes, COPD, 30+ BMI obesity, Alzheimers, opioid dependency, general decrepid-ness, etc.

    In other words, the mortalities are all those folks who are useless to the economy. The young, the strong, and the healthy who are capable of lifting, toting, grinding, shoveling, and otherwise doing useful work do not get the virus — or if they do, they have minor symptoms and get over it.

    What a boon to the Economy! The slackers die off while the Workers toil on. Those clever Chinese! Just what the Supreme Leader Commie Premier ordered!

    That’s why from now on I’m calling Covid19 the Chinese Designer Virus (CDV).

    It’s unfortunate for me that I am one of the slackers, but I understand the Purpose of the Program. I knew something like this was coming. Even so, I’m hiding out and wearing a space suit. You’ll have to hunt me down and shoot me, you damn kids.

  8. Every so often, one hears interstate truckers blaring their trucks’ air horns, the reason being is to keep the elephants off the highways. Must work because has anyone witnessed a vehicle/elephant collision on them thar highways? Nope. Correlates real well.

    Given that, I think I’ll try my hand at projecting COVID-19 infection and death statistics.

  9. Tiny Tony Fauci is the Marty Byrde of medicine and the license free Brix is Wendy Byrde.

  10. When it’s all said and done (in a year or so?), probably comparing the excess morbidity in Sweden (very little oppressive measures taken, everything is just recommendations by their health officials) to the rest of the Nordics (stricter policies in place) will give some insight to as how the social distancing might have worked — or not.

  11. HOW HONEST IS THE COVID FATALITY COUNT?

    https://www.powerlineblog.com/archives/2020/04/how-honest-is-the-covid-fatality-count.php

    “Dr. Scott Jensen is both a physician and a Minnesota state senator. Yesterday he was interviewed by a local television station and dropped a bombshell: he, and presumably all other Minnesota doctors, got a seven-page letter from the Minnesota Department of Health that gave guidance on how to classify COVID-19 deaths. The letter said that if a patient died of, e.g., pneumonia, and was believed to have been exposed to COVID-19, the death certificate should say that COVID-19 was the cause of death even though the patient was never tested, or never tested positive, for that disease.

    “Dr. Jensen found this to be irregular and contrary to the usual practice for filling out death certificates. It seems intended to inflate the number of Wuhan flu deaths; it is hard to see any other potential purpose. The most entertaining thing about the exchange is the shock expressed by the interviewer. {NOTE: Video in link.]

    “Can we trust the COVID fatality counts that we are being given daily? I think it is fair to say that powerful forces would like to maximize the apparent impact of the disease.

  12. Sheri,
    How come we never build field hospital for regular flu. Regular flu is simply not as contagious, and the morbidity is much lower.

    Whatever what other aggravating factor someone might have. In the end, People simply suffocate to death. This is cause by the virus. Hence the virus is the cause of death.

    1-What you moron don’t understand is that people who are healthy die from this virus. New York has already lost firefighters, police officers, nurses, paramedics and doctors to this virus.
    2- people who died in early January might have died of the virus but the death was attributed to the flu.
    3-The flu is not an efficient agent. Its incubation is only a few days and most people are sick for about a week .Young people can need a ventilator for up to 2 weeks. For this virus people are sick for weeks. The UK Prime Minister has already been sick for more than 14 days now.
    4- With all the contagion you have in the USA. Try to imagine how it would be if there were no stay at home order.

  13. Dave –
    We (both you and I) can only hope your last days and weeks are spent isolated from family and friends, with your only real human interaction coming from folks in space suits.

  14. Good ol’ Gavin Schmidt provides caveats about the efficacy of Covid-19 models but is noticeably quiet when it comes to climate models.

    Is that irony or what?

  15. Data tables from the Italian Statistics site:
    https://www.istat.it/it/files//2020/03/mortalit%C3%A0-per-giorno-periodo-01_01-30_04-anni-2015-2019-totali.zip
    https://www.istat.it/it/files//2020/03/Tavola-sintetica-decessi.xlsx
    From these very tables the total death rates for Lombardy region for the month of March from 2015 to 2020 are:
    2015 2016 2017 2018 2019 2020
    8552 8037 8059 8523 8543 8587
    I don’t quite see the reason to panic. Bergamo, the superdeath corona city, is included in Lombardy!
    For the US:
    https://www.cdc.gov/flu/weekly/weeklyarchives2019-2020/data/NCHSdata13.csv
    Death rates for the first twelve weeks of 2020 are utterly unremarkable, both for Pneumonia/flu and for total death rates compared to previous years. If there were no tests for the corona, it would be a normal flu season, with zero warning that things are maybe not ordinary! Total death rates actually dropped from ~50000 per week to only ~40000 per week since the lockdown, probably due to less driving, and emptied hospitals, reducing iatrogene death rates.
    I am convinced, that most corona cases are just relabeled pneumonia/flu cases as otherwise they would show up as increased total death rates. It is hysterical moronic imbecility world wide.

  16. “How come we never build field hospital for regular flu.”

    Logical fallacy much? Meant as rhetorical? Quite typical from the canard francais from the land of Nunavut!

    Hysterical or insane over-reactions to a stimulus are not evidence of the strength of the stimulus, mon frere!

    “How come we don’t scream hysterically when we see other spiders?”
    A little girl screaming her lungs out because she saw a spider is not evidence of the deadliness of the spider.

    “How come we don’t build Armageddon Time Ark Bases for other threats?”
    A doomsday cult built Armageddon Time Ark Base for extra-terrestrial spaceships to land and rescue them when Armageddon comes. Their over-reaction is not evidence of the seriousness of anything at all.
    http://www.atabase.info/

    Not sure what the proper name is for your logical fallacy, but it is surely an excellent example of one!

  17. There was no such thing as ‘Spanish flu hysteria’ in 1980, no internet, twitter (thank heavens), mass air travel, Fauci, shelter-in-place, etc.

    Yet, 50million people in Europe died. I mean, it was just a flu, right? Meanwhile, in the USA, St. Louis and Philadelphia took a different approach to mass gatherings and the data clearly show which one was a smart move. If that same strain was going around today, it wouldn’t be noticed, but it was a novel thing then, not recognized by our immune systems.

    For every person that is over-counted as a ‘victim’ of COVID, there is an unreported one that died of COVID, but wasn’t tested to be confirmed. There are many such reports (from Europe at least) of people being found in their houses after not responding to the calls from friends and family. You really think they were/are wasting tests on corpses when this wasn’t even a ‘thing’?

    This new virus has all the dangerous elements: long incubation, low overall mortality, dangerous symptoms, unpredictable targets, etc. One silver lining is that when the vaccine is developed, it will be much more effective than what’s used for seasonal flu, as it doesn’t appear to mutate much.
    Thanks

  18. Kent,

    You realize that even with mitigation the corona virus will kill more Americans than the flu this year and in much fewer months

    Rural areas are getting infected in the USA soon it will be every American for itself and the end of an Empire precipitated by the election of the idiot in chief.

  19. Trebtch

    So you compare total death in a month to death of one illness.

    You realize that there were has many death in one month from 1 illness in 1 region under quarantine than in the same region where people die of any other illnesses and accidents when it is not quarantined.

    The number of dead would have been a lot worse if there had not been any quarantine.

Leave a Reply

Your email address will not be published. Required fields are marked *