Announcement: I wrote this Class for a general audience. Anybody can read it. And should if they are interested in Science. There’s only a little notation, which can be ignored by most. What applies here, applies everywhere, not just in Science.
I got word from Dr Roller Gator that Ehrlich, whom I abuse below, up and died on me right after I finished this Class. I don’t know if I can forgive him for this. But since I didn’t know he was at his final judgement when I wrote this, I’m leaving it as is.
The key lesson is that judgments are not part of science. They are something we bring to science. And that when one man sees a model as good which another says is bad, it is because they must be using different judgmental criteria. The goal is always to find these criteria and discuss them before discussing the models.
Video
Links: YouTube * Twitter – X * Rumble * Bitchute * Class Page * Jaynes Book * Uncertainty
HOMEWORK: Find your own time series with trendy trend lines. See if they suffer from the Deadly Sin of reification.
Lecture
Our title is too narrow, but it is (I hope) catchy. Really what we want to know is what separates good from bad models. All models, formal and informal. Which all of us use constantly, only maybe you didn’t know it. Since Science uses so many formal models, examples come ready made. And, when it’s possible, they are more amendable to quantification.
It isn’t always possible, or even desirable, that we quantify goodness, or even probability. We don’t need, and can’t have, measures on all things. The desire to quantify the unquantifiable is part of scientism. Which leads (yet again) to vast over-certainty. But enough about that today.
Reminder, all probability, which is to say all logic (probability and logic are one), is of this form or schema:
Pr(Y|M)
where Y is what we want to know, our “conclusion”, and M is all the evidence we consider (called only “premises” in logic). I use “M” for the obvious reason: because another name for our evidence is model. This is what all models are: the evidence we assume. Nothing more.
I also, contrary to custom, do not distinguish between “model”, “hypothesis”, “theory”, “law” or any other word you care to use for the evidence you assume and its relation to Y. To me, they are all the same.
We will now be judgmental. Being judgmental is good. All of us ought to be judgmental. We want to judge how good evidence M is in relation to Y. To do that, we need a Judgmental Function. Which need not be quantitative, but often is in Science (if you can’t follow notation, ignore it, and focus on the meaning, which you ought to be doing anyway). In other words, we want:
J(Y, Pr(Y|M)|W),
where we compare the real Y, the Y that happens in Reality, with our model of Y, which is Pr(Y|M). The W is where the magic happens, defined in a moment.
We use the function J when either Y or Pr(Y|M) is not quantified. When we can quantify both, it’s more common to speak of a Score. In those cases we’ll use S. I.e., in the same way:
S(Y, Pr(Y|M) |W).
That W means Worth. It, like M, is a list of premises or conditions that are meaningful to you. But not necessarily to anybody else. You might have read me before saying that a model might be useful (or worthy) to one man, and completely useless, or even harmful, to another. Today I prove this.
Like in logic, if M is itself a necessary truth or truths, then Y will be, too (or Y will be false). But we can get local truths with ease, which you recall from when we started, is when M isn’t entirely true, but where we deduce Pr(Y|M) = 1 or Pr(Y|M) = 0. Local truths abound. If M = “All feminists have a sense a humor; and Jim is a feminist” and Y = “Jim has a sense of humor”, then Pr(Y|M) = 1, even though M is clearly false.
It’s the same with W. If W is a necessary moral truth, say, then our judgment of M will also be a moral truth. Whereas, if W contains some measure of unproven assertion, then so will our judgement of M be unproven, even though it can be locally optimal.
Some examples will cement this. I want to first clarify that you won’t read this kind of thing elsewhere. As far as I know, this terminology is not widespread.
Let Y be something like “Mass starvation and death”, where by “mass” I mean at least millions, if not more. And let M be a model of “climate change” or some equivalent environmental degradation, like that caused by “overpopulation.”
Paul Ehrlich has said, and has said many times, things like “Pr(Y|M) = 1”, or close to 1. He said “hundreds of millions” would starve to death in the 1970s. I mean, he said that in the 1970s about the 1970s. Population growth was going to do this. That was his model. He has since repeated many variations of this model, adjusting the premises of the models in various ways, but always leading to high probability, if not certainty, of great doom (his Y).
How shall we judge his model? Or models, really.
One way is to declare a W that says a model must have made a prediction which matched Reality to be good; perhaps not exactly, but anyway “close”. The closer the match between model and Reality, the better. Good models are accurate, bad models are inaccurate. This is a moral judgement. Let’s give the subscript A to our W so we don’t lose track: W_A says accuracy is good, inaccuracy bad.
Then
J(Y_D, Pr(Y_D|M_E) | W_A) = something.
But what?
That “E” subscript stands for “Ehrlich”, and “D” for “at least millions dead.” What J equals depends on how you phrase the function. Later, with Scores, we’ll use the convention that “close” is better, and that “better” means lower scores, which means a perfect score would be 0. But this is just convention, and we could have taken better to mean higher scores.
If we could quantify Ehrlich’s model, and we take the logical contrary of Y_D, which we might call antiY_D, then
J(antiY_D, Pr(antiY_D|M_X) = 1| W_A) = 0,
or something real close to 0, where “X” is an anti-Ehrlich model, i.e. one which makes prediction the opposite of Ehrlich. Because the opposite of what Ehrlich said would happen happened. Which makes:
J(antiY_D, Pr(Y_D|M_E) | W_A) = Ehrlich bursts into flames, the ground opens and swallows him as he falls screaming into the pit [again, I wrote this prediction before knowing he died],
or whatever numerical equivalent you want to give that. His model was so bad, conditional on W_A, that even politicians in “Our Democracies” blush when considering his performance.
But this W_A is not the only W possible. Many others are plausible.
Let W_U represent academic quality. The “U” is for university (we already used A). A model is good conditional on W_U if it garners praise and glory from academics and the intelligentsia. It is bad if it excites opprobrium and insults.
Ehrlich might not have made Top Man, but few scientists have been as feted, celebrated, awarded, caressed, remunerated, envied by others, and rewarded as Ehrlich [was], all because of his models.
That means
J(antiY_D, Pr(Y_D|M_E) =1| W_U) = 0 or close to it,
if we stick with our convention that smaller is better. But it also means
J(antiY_D, Pr(antiY_D|M_X)=1 | W_U) = harsh braying calumnies, insults, charges of being a “denier”, lack of promotion, grantless, ignored, kicked to the curb
for the opposite of his predictions.
When considering men like Ehrlich, the phrase is “many such cases.”
And, it turns out, for many men to come after Ehrlic. Here’s a headline Dr Roller Gator saved for me:
Notice very carefully that “premature.”
Ehrlic’s predictions were only bad when conditional on W_A, i.e. where accuracy counts. But his model was perfection itself conditional on W_U, where prestige counts. The person who wrote this blurb held with W_U. And he or she is obviously sticking with Ehrlich’s model.
Now this is only the first steps into the are many call forecast or model validation. What haven’t done much more than introduce a couple of concepts. We only know we can assess models M in relation to moral judgements W. Models themselves may be conditional on other evidence, downstream, as it were, from the models themselves.
But we have proved that models can be useful to one man and useless to another. Usefulness is in the eye of the beholder. Which is why many can demand “Follow the Science!” while others scoff in dismay.
We can quantify these things, which can be useful. The “Y” don’t have to be dichotomous, but can be, well, anything, any proposition. We have to learn how this all fits with calibrations, sharpness, proper scores, “ROC” curves (and why I don’t like them), skill and many other things. Please stick around.
Here are the various ways to support this work:
- Subscribe at Substack (paid or free)
- Cash App: $WilliamMBriggs
- Zelle: use email: matt@wmbriggs.com
- Buy me a coffee
- Paypal
- Other credit card subscription or single donations
- Hire me
- Subscribe at YouTube
- PASS POSTS ON TO OTHERS
Discover more from William M. Briggs
Subscribe to get the latest posts sent to your email.
