Models Only Say What They’re Told To Say

Models Only Say What They’re Told To Say

https://twitter.com/DrEricDing/status/1260234198578802690

“Trust science!” says Deng (lower down in the thread).

Isn’t science self-correcting?

“Yes, that’s why it’s so great.”

Then science is sometimes wrong?

“Well, yes, sometimes.”

So how do we know this isn’t one of those times it’s wrong?

Trust Fauci!

——–

(Incidentally, before we start this, if you wondered how leftist singularities happen, gaze upon that Harvard guy’s picture and wonder no longer. )

Friends, I don’t ask it often, but please share this article as widely as you can.

Headline: Updated Model Predicts COVID-19 Peak In Late-July With SAHO Extended Through May; 25K Deaths Possible

Here are the Minnesota daily deaths, with our naive model overlaid—-which are you free to ignore. As of Tuesday night, about 9 PM, there are 614 total reported deaths in Minnesota. Our naive model predicts 800 in total, assuming no second, third, fourth, fifth, and more waves.

You can see—forget our model—-that deaths if deaths are on the way down, they at least aren’t increasing, just as you’d expect in any typical outbreak. Weather is slowly improving in Minnesota, as it sometimes does.

Here’s what the article says: “An updated model from the University of Minnesota and state’s health department is predicting that COVID-19 cases will peak in late-July with 25,000 deaths possible — if the stay-at-home order is extended until the end of May.”

Twenty-five thousand more deaths even if the stay-at-home “order”—memorize the word—is obeyed. Even if.

In Scenario 5 [one of several model scenarios], the stay-at-home order is extended for all until the end of May. With that happening, the model predicts that the COVID-19 peak will happen on July 27, with the top intensive care units (ICU) demand being 4,000 and 25,000 possible deaths.

Another estimate, Scenario 4, predicts that if the stay-at-home order is extended by a month into mid-July, the peak would occur on July 13 with 3,700 as the top ICU demand and 22,000 possible deaths.

All right. Here’s what the great leader of that state said about models.

In previous briefings, Gov. Tim Walz has asked Minnesotans not to focus on specific numbers, but rather focus on when the peaks might occur.

“Modeling was never meant to provide a number,” Governor Walz said on Wednesday. “It was meant to show trend and direction, that if you social distance you buy more time.”

No!

Social distance working is an INPUT to the damned models. You can’t point to model output and say, “See? The model says social distancing works!”

No shit, you *&&^^&)**!

I have seen this mistake made so many times in this crisis that I am well on my way to a stroke.

The models reported on—just like Ferguson’s and everybody else’s—are built saying social distancing reduces death. This fact is an integral part of them. It’s not a “discovery” of the models, it is a condition in them. The models had to say social distancing worked because they started with the premise social distancing worked.

You cannot “discover” social distancing worked via any model. You had to have built in that possibility in the first place. You knew in advance that it worked because that’s what you told the model.

You cannot run the model, wait for the output, run to your boss and say “Look at this, chief! The latest model says social distancing works!” If your boss had any sense he would slap you and say, “Didn’t you write that code? And didn’t the code say somewhere that social distancing worked?” If there was any justice, you’d join the ranks of everybody else fired because of the panic and be forced to stand at the end of the bread line.

I don’t know how else to say this. I am obviously failing to make the point. I admit to being unable to make it so that it sticks.

Why doesn’t it stick? All I can figure is that the Deadly Sin of Reification strikes when people think about models, even their own. Excuse me: computer models.

A scientist creates a model, or a civilian looks at a model created by some smart person, and, somehow, the model takes on a life of its own. It becomes and replaces reality, this model, at least in part.

Since the model is now “reality”, we can look at it like we do Reality and pretend to come to an understanding of how the world works. The model’s workings take the place of Reality’s workings.

This is all wrong. Yes, a model can be so complex that if you push lever A, the “unexpected” result X is churned out. X is only “unexpected” because of limited intelligence of the model writer. Further, X can only come out because the model was built with that possibility. This must be so. If it wasn’t, then X would never come out!

Models do help simplify Reality. They do aid in thinking about how things work, what causes what. They clarify thought. But they never, not ever, say what they are not told to say. Not ever. Never.

(No, not even when you build in “randomness”. Every piece of every model does only what it is told, and nothing more.)

Learning what to tell models what outputs to make (given what inputs) is the real point of modeling; for that is when we can make skillful predictions. That is when we have a grasp of cause, or at least of correlation.

Yet I insist that you can never get out of a model what you do not put in. That means whoever built the models predicting insane numbers of dead bodies in Minnesota unless the stay-at-home “order” was obeyed knew what the outcome had to be in advance. In the sense they knew social distancing would lead to predictions of lower deaths. Even stronger, they must have been advocates for this position.

That, or they have no idea how models work. This means the modelers are incompetent or that they are true believers. Their only possible hope of proving competence is if Minnesotans start keeling over at magnificent rates.

To support this site and its wholly independent host using credit card or PayPal (in any amount) click here

28 Comments

  1. Gary

    Humans are very bad at assessing risk, knowing themselves, and telling the truth.

  2. Michael Dowd

    Wouldn’t be helpful Briggs if you could obtain a copy of the model used by Dr. Fauci so you could point out the fallacious assumptions, etc? Is it possible to obtain this information?

  3. Briggs

    Michael,

    It’s been done. Didn’t I link to it yesterday? Maybe not. I have 82 million tabs open and can’t remember what I linked and what I didn’t.

  4. JR Ewing

    Two things I don’t understand about states that are still hiding under the bed:

    1.) How can people still credulously believe these predictions after the past two months? Models can – and should – work both ways. When the model verifies, that means you got some input correct, but when they don’t verify, that logically MUST mean *you got some input some input WRONG.”

    The reason that these model predictions keep busting is because this is obviously not as dangerous as we initially ASSUMED it was. PERIOD. END OF STORY. We now have DATA that shows that it’s not as dangerous to the general population. Antibody tests that show a preponderance of asymptomatic and minor cases. Hospital data. The fact that South Dakota and Georgia and Texas and Florida and Sweden haven’t plunged into the fiery abyss. All kinds of data that is being ignored and not plugged into the models.

    How can so many people in these states still accept the breathless bullshit forecasts that are still being peddled on TV? It makes no sense. The only way any of this is still working is because there is some portion of the population who willingly wants it.

    2.) The other thing I don’t understand is why there haven’t been riots among the nonbelievers in these states where the politicians refuse to acknowledge reality? I know Minnesotans are “nice” (It’s true, I have experience with them), but they’re still humans and they should have a breaking point. Michigan too. At some point, you’re obligated to push back. Hard. Ask Thomas Jefferson. And I’m not talking about waving a bunch of handmade posters out on the lawn.

    I honestly don’t know how my sanity is going to survive the next few months. Talking about keeping schools closed and canceling youth sports “to keep kids safe”, when there is no data at all that suggests there is a danger to them. My son is an 8th grader. His window in life to play football is tiny. He has 5 years left, maybe 9-10 tops if he can go to college. Why take 10-20% of that time away from him that he’ll never get back, when there is no threat? And what about high school kids who missed their prom? Or graduation? Or those who will miss their senior football season?

    It just makes no sense.

  5. DAV

    Already saw the Tucker re Fauci last night. Can’t help hearing Bullwinkle saying “This time fer sure!”

    Then there’s the Dem multi-trillion dollar proposal. Doesn’t anyone study economics anymore? If everybody gets $1000 then prices will rise to meet the money supply. We’ll become like Thailand where 1 Baht is worth just a little over 3 cents.

  6. Sheri

    JR Ewing: People watch the MSM that NEVER, EVER reports any of the things you mentioned, even on a local level. If person’s god is the MSM, they BELIEVE no matter what. It’s a very widespread phenomena.
    You answered your own question on how they can take 10 to 20% of your son’s opportuinty: NO ONE stops them. The Michigan Militia is finally getting louder and louder, blocking the arrest of a 77 year old barber who dared to reopen after Gretchen the Witch said no. I’m not sure how that turned out. If we want this stopped, the news media needs to be dragged out into the streets and flogged. There is no other cure. Without the MSM, Gretchen the Witch and all the other governors lose their power.

    DAV: AOC has a BA degree in economics. I do not see that helped.
    My husband tried to explain the cost increases once to a union member who refused to even listen. In spite of layoffs and price increases, idiots still want a $15 minimum wage. This live in a fantasy world, pure and simple.
    Yes, as wages and handouts rise, money loses value. A wheelbarrow full of money was needed in Venezuela just to buy a loaf of bread.

    If we are not to focus on the numbers, why even bother to provide them? I think we all know the answer to that, don’t we Lying Tim?

    Actually, we are told the model DOES prove social distancing works because if we take out the social distancing, bad things happen. Are you saying this might be trick and they are fooling the mathematically, statistically ignorant out there, not to mention feeding those who love gloom and doom?

    I’m sure the Deadly Sin of Reification is a major factor and is what allowed this “pandemic” to really be a destruction of economies and human beings. To people this the idiotic, 4000 to 22,000 is a real number. It’s insane–the range cannot even resemble reality or have any scientific meaning whatsoever. It’s a wild guess, and not a SWAG either. I have the same problem trying to explain this. People see a computer printout and that is reality, nothing else matters. Electronic soma with a truth value beyond reproach…….

  7. Matt Fisher

    As a Minnesotan I’m glad to see our models getting some attention. Some things to note about our model in case you’re interested:
    1. The original release (March) predicted 74,000 deaths in MN without SAHO and 50,000 deaths with SAHO. The fact that we’d still have 50,000 deaths with a broad SAHO was not mentioned in the early press conferences.
    2. Recent revisions to the model showed that extending the SAHO for everyone increased the death toll slightly compared to a targeted SAHO for at-risk populations. This is presumably due to a delay in reaching herd immunity which puts the at-risk population at greater risk.

    So the Minn model predicts wildly high mortality. Those numbers are used to justify SAHO. The model then shows extending broad SAHO actually increases total mortality. That output of the model looks to be ignored.

    All I can say is that as a non-believer in a place like this you’re in a clear – and I think pretty slim – minority. In response to JR – I agree it’s largely about the MSM coverage. There is a complete lack of “putting things into perspective”. The above-the-fold headlines in the StarTrib are almost always anecdotal fear-mongering stories. Zero perspective. Then you dig into the OpEd and local news sections and there’s at least some balanced coverage. But what they put above-the-fold is all about fear and that’s what the public is buying.

  8. Joy

    Dear Sir/madam,
    “models only say what they’re told to say”!
    Thank you for illustrating that so very well. over the years.
    To me, who knows nothing, they’ve always looked like fancy graphs.
    Some people around here though have managed to show where they have uses. For a very long time I secretly didn’t believe they had ANY use at all.
    Numbers tell you what you need to know and understanding mechanism. If things get too complex for even smart bears, then models have use in an honest pair of hands.

    Lay people shouldn’t be allowed to touch models if lives depend on it or big money.
    Like chain saws and heavy machinery.
    PS, now I’ll read the article!

  9. c matt

    The fact that South Dakota and Georgia and Texas and Florida and Sweden haven’t plunged into the fiery abyss

    I get that these locales have come out ok, but what works in one place, may or may not work elsewhere. Which is why having a national, or even state-wide one size fits all policy is stupid. There are counties in SD and Texas where the cattle out-number humans. If NYC or Chicago want to lock down, let them. You can’t take a pee without hitting four other people in those places. Where population density, weather, demographics, etc. don’t warrant lock down, it shouldn’t happen. Besides, anything that makes NYC or Chicago suffer, I am all in.

  10. Rudolph Harrier

    Some observations from outside the Twin Cities:

    -In my estimation about 1 in 10 people are wearing masks in stores. Practically nobody wears them outside of stores.
    -Parks are packed, especially on weekends. This includes city parks, county parks and state parks. The only parks that are still pretty sparse are those that are at least forty minutes from a decently sized city.
    -Tim Walz said that the next phase might be to allow family gatherings. These gatherings have been going on for weeks, sometimes including dozens of people. I don’t know of any incident where the police were called on such a gathering outside the twin cities, though I have not looked extensively into the matter.
    -Similarly, many businesses (beyond just grocery stores and the like) have allowed small numbers of customers inside. Dry cleaning and the like have been doing curbside business for weeks, despite that supposedly only being allowed starting this week.
    -Traffic on the roads seems to be about the same as it normally is, with the exception of “rush hour” traffic not being as bad (not that it really slows down that much at those times normally).

    If Walz extends the stay at home order another time, nobody outside of the Twin Cities is going to listen to it. (Well, maybe nobody except our Catholic Bishops).

  11. Paul of Alexandria

    I must respectfully say that I disagree with this article. Models exist in order to allow us to predict future events. They do not, usually, depend in any way on “randomness”, unless you’re modeling noise or component variation. They do rely on complex equations that simulate how some corner of the universe works. Engineering depends on models and has since the ancient Greeks. As the old saying goes, “if you don’t have the numbers” it’s not engineering, and nothing gets made until it’s first been simulated, even if only through pencil, paper, and Kirchhoff’s Laws! We don’t put into models that X, Y, or Z “works”, we put in how the universe is known to work, as best we can. If the model is good, the result will be a pretty good approximation of how the actual device will operate – or not operate, as the case may be.

    I rather doubt that any modelers put in that “social distancing works”. They probably did put in some known factors regarding the infection rate of a Coronavirus vs distance between individuals and population density, and if these assumptions were wrong than the models should be corrected. But I rather doubt that any explicit assumptions about “social distancing” were included, unless the pandemic modelers, like the climate modelers, are hopelessly corrupt.

  12. Dave

    It’s not controversial that distancing lowers infections compared to an unknown counterfactual with no distancing. How can an infected person infect an uninfected person, unless they cross paths? The reason infections don’t stop immediately with “shut downs” or distancing is because lots of people are still working and interacting; fewer than before, but not none. Witness meat packing plants, for one recent example. The real issue is that standard epidemic models don’t account for people changing behavior voluntarily.

    The standard SIR model supposes that people don’t change their behavior on their own. They feed it “R0” and pretend that people won’t ever change their behavior (modeled rates of infections only decrease as a result of how many people are immune or dead). This leads to over forecasts, when they see high contagion rates early on before people learn to take precautions and then project it everywhere for the rest of time. That’s why they insist on government action, because the rate of infection is exogenous to the model. Actors in their models don’t have agency.

    One the other end, you have this model, which forecasted “~160,000” total cumulative reported cases in the world: https://www.wmbriggs.com/post/29734/

    So my conclusion is that pandemics are generally hard to predict.

    I’ve seen an economist add some more realistic endogenous behavior here:
    https://johnhcochrane.blogspot.com/2020/05/an-sir-model-with-behavior.html
    https://johnhcochrane.blogspot.com/2020/05/dumb-reopening-might-just-work.html

    At the very least, it’s a better mental model of what happens, with people reacting to information with precautions when death or infection rates are high and easing up when they are low. Although, I still wouldn’t trust them to make precise forecasts.

    In fact, the best forecast I’ve seen is from March 17th, when I predicted that Briggs would predict a cumulative total of “4.5M” worldwide cases (https://www.wmbriggs.com/post/29830/#comment-184547), which is exactly what he did yesterday: https://www.wmbriggs.com/post/30804/

    Sure, I was off by a few weeks for *when* he would make that forecast, but sure beats all of the forecasts which have been off by an order of magnitude…

  13. Rich

    I think I first saw this in How To Lie with Statistics but I had to look it up:

    “Harold Cox tells a story of his life as a young man in India. He quoted some statistics to a Judge, an Englishman, and a very good fellow. His friend said, Cox, when you are a bit older, you will not quote Indian statistics with that assurance. The Government are very keen on amassing statistics—they collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But what you must never forget is that every one of those figures comes in the first instance from the chowty dar [chowkidar] (village watchman), who just puts down what he damn pleases.”

    It occurred to me that everyone (sort of) speaks confidently of when lock-downs started but the experience in the UK suggests that a mere Government announcement tells you little. The weekend after our government announced lock-downs thousands of people, crowded together, walked up Snowdon because it was sunny. British people: herding cats.

    Rich

  14. John B()

    Both Paul and Dave both brought up Infection Rate or R0. The UofMN model used an R0 of 3.87. Why? Is that what we came up with when Covid was first active?

    It’s a corona virus. The common cold is usually a rhinovirus, mostly caught during the fall and the spring (wonder what its infection rate is before during, inbetween and after those seasons). Deep in winter or in colder climes, the common cold as corona virus is active (infection rate? before? during? after? What causes a particular infection rate?

    Some years we find it a nasty cold season (we’ll leave the flu out of it). What affects this infection rate? In the summer, we’ve got the ernterovirus (a different ugly animal according to an ancient commercial). Infection rate before, during, after? Is the lethality of a virus affected by the season?

    Look at the graph of the MERS during its peak in 2014, an uptick in Feb, then accelerating through March, and through the roof in April, then quickly fell through the rest of April, May and tail in June. Pretty sure that even in Minnesota there should be no huge peaks in July. Oh! Yeah! That was MERS-CoV.

  15. “Then science is sometimes wrong?
    “Well, yes, sometimes.”
    So how do we know this isn’t one of those times it’s wrong?”

    Minor (ok actually major) point: science doesn’t claim to be always correct (unlike religions). You wouldn’t “know” until data comes in, but you wouldn’t know that from non-science ways either, nor would those non-science ways be open to correction, nor some be open to even looking at data. For example, many religions.
    Cheers,

    Justin

  16. I don’t know much about models. But it seems to me that one component that would be crucial to put into them is accurate data on the ground–that they should begin with this data. Currently the effect of the shutdown on jobs has hit the hospitals, only second to the restaurant industry. “Flattening the curve” is sinking the hospital industry–not to mention jeopardizing the life and health people whose conditions are considered “non-essential” .
    Also I don’t know if remaining six feet apart prevents the virus from jumping from one person to another. What it does do, however, is allow for satellite recognition of the face.
    And since, in part, we’re talking about scientific accuracy, I noticed that medical personnel are wearing masks foe airborne microbi, while the public is expected to wear masks suited to droplet infections. Which is it? Also, old school, we were taught that the latter masks lasted approximately 20 minutes, after which the moisture of the breathe rendered them useless. However, leaving them on for a long period of time creates a moist environment that I assume would make one more susceptible to infection.
    And then there’s the problem of using sanitizer all over the place. Don’t we want to live in a microbial environment so super bugs don’t develop?

  17. someone

    The code is here:
    https://github.com/mrc-ide/covid-sim/blob/master/src/Update.cpp

    I agree completely. It seems incredible that people do not realize that the assumptions of house confinement, square of distance and mask types are the ONLY factors coded into the models. Therefore they are the only possible actions that could affect the model numbers as policy,

    There is no other variable that is represented. The code does allow for a percentage of Immunity at start but this is set to 0. (It is also weird that it has only 4 place types (primary school, high school, university and offices – no nursing homes or shops or outdoor places – so it seems it was created for a virus centered around the young)

    It is also strange that supposedly for code that is 10 years old it has the idea of Digital Contract Tracing built in.

  18. F4F111Col

    You missed the most ironic (and revealing) part, Briggs. In the picture, Mr Feigl-Ding and his progeny are giving the Communist clenched fist salute, as is routinely done in the PRC. Yes, even in nursery school, the little kids are strongly encouraged (and you know what that means in CHICOMM land) to do so.

Leave a Reply

Your email address will not be published. Required fields are marked *