Reminder: The Thursday Class is only for those interested in studying uncertainty. I don’t expect all want to read these posts. So please don’t feel like you must. Yet, I have nowhere else to put them besides here. Your support makes this Class possible for those who need it. Thank you.
It is the one which says what the model does for you, and not necessarily anybody else. Do not fall prey to the Deadly Sin of Reification. All judgements and probability are conditional
Video
Links: YouTube * Twitter – X * Rumble * Bitchute * Class Page * Jaynes Book * Uncertainty
HOMEWORK: Read!
Lecture
Last time we learned that there is no, and can be no, fixed standard for judging a model’s worth or goodness, save whatever universal or necessary moral truths exist and are applicable to the situation. Which led to our conclusion that the same model may be valuable to one man and worthless, or even harmful, to another. Paul Ehrlich, we saw, benefited greatly from his modeling efforts. A rank understatement: he positively soared. Yet his models had the same allergic reaction to Reality as do feminists with men.
That this is so is yet another—our quiver was already overflowing—argument against P-values or any kind of “testing”. Because those tests make decisions, one-size-fits-all decisions about model goodness and value. Which is why they fail so spectacularly when somebody’s Judgmental Function does not match that of these fallacious tests (the fallacy comes from the probability mistakes and not the JF). Hence, at the least, the reproducibility crisis and the vast pandemic of over-certainty in Science.
Recall our Judgmental Function:
J(Y, Pr(Y|M)|W),
where Y is the proposition of interest, “Pr(Y|M)” the model probability statement, M the model, and W the set of worthiness propositions that make sense to us, and perhaps nobody else.
And that is the subject, and conclusion, of today’s lesson. The best measure of model goodness is always how valuable the model was to you, in whatever situation or situations the model was employed.
Your Y = “The car keys are on the hook in the kitchen.” Your model M = “The keys are usually on the hook”. Your Pr(Y|M) = very likely (not a number). You go to the hook: the keys are not there. How good was the model? Depends on your W. If it contains anything about not wanting to search obscure places first, then the model is surely reasonable, even though it failed in this instance.
If we were to make the Judgmental Function into a Score, it has the same functional form (as we learned), only all these symbols now have numerical values. Let’s make some up for the exercise. How about Pr(Y|M) = 0.98? Only now M must contain at least tacit propositions that allow us to assign this number.
Pause and make sure you grasp this. This is one of the two main fundamental lessons of the Class. That all probability is, and must be, conditional on the information we supply.
Scores are often distances, which formal mathematical definitions. A popular one is the squared distance between the probability and Reality. This has, as mathematicians are wont to say, “nice properties”. One of which is that the score is always positive. Our Reality-Y is now dichotomous, which is got by applying an “indicator function” to the outcome. In other words, our Reality-Y = 0: the keys were not there. Formally, we’d write
S(I(Y) = 0 , Pr(Y|M) = 0.98 |W = “squared distance”).
The answer is (0 – 0.98)^2 = 0.9604.
Is that helpful? Not too much, no. But let’s see where it goes. The maximum Pr(Y|M) can be is 1, and the minimum 0. M might be “The keys are always there” in the first case and “The keys are never there” in the second. There are obviously other choices for M.
If Pr(Y|M) = 1, and I(Y) = 0, then S = 1. That is the upper bound. The lower is clearly 0, which happens when I(Y) = 1 and Pr(Y|M) = 1 or when I(Y) = 0 and Pr(Y|M) = 0. The latter is not especially interesting for many models. For instance, M = “The keys are never in the butter dish” (and they are not). Or “on the moon” or so on and on. Remember another central lesson: there are always an infinite number of models that can be had, and that fit arbitrarily well.
Our goal is thus never scores, but cause. We want, and always want, the full explanation for the proposition of interest. Here, where are the car keys and how did they get there?
In any case, this score with S = 0.9604, given this W, says the model is bad, because S is near its maximum, and you recall from last time that higher scores are worse by convention (quite quite quite obviously we could reverse this whenever we like).
And that’s where it would rest if we picked this W. We’d reject M and not use it again, perhaps. Or maybe we would if it’s successful enough times. How can we tell? There is nothing that insists we must use a unidimensional Fudgment function. We could keep this W and add to it a dimension of search costs. In the end, if you have to make a choice whether to keep the model or modify it, you have to weight the different W you consider. Any number of considerations can come into play, including what happens “downstream” of any model if you go on using it.
There is no general solution. If there were, then Science, and life itself, would be “plug and chug”. Everybody would always know the right thing to do, and the wrong thing not to do, or they would learn it fast. Sometimes this is so, but like we learned last week, that is when we know W is a universal or necessary moral truth and we know it fits our situation. For the rest, when there is ambiguity, as there is in most things, we are left groping.
I harp on this because it’s the same as with P-values. Once we begin the math of scores, we get stricken by their mathematical allure. We’re sucked in. The scores become real. They are numbers! The P is so wee! It’s math, and a lot of hard math, too. We must defer to numbers. In our culture, it’s almost impossible not to. They are so scientific. And scientism is our state religion (after Equality).
All judgments are conditional, just as all probability is conditional. There are no unconditional judgments. There is no unconditional probability. Our premises for either are always there, even when we don’t acknowledge them, and they are always the most interesting part of any problem. They are often forgotten, though, when numbers are involved. It’s a terrible thing. The model becomes realer than Reality. This is the Deadly Sin of Reification. I am telling you now that you will fall prey to this.
But you will. We all do. Be watchful.
Next time we begin the math in earnest, which in spite of all these warnings, is useful, just as it is with probability. As long as we know at all times what we are doing.
Here are the various ways to support this work:
- Subscribe at Substack (paid or free)
- Cash App: $WilliamMBriggs
- Zelle: use email: matt@wmbriggs.com
- Buy me a coffee
- Paypal
- Other credit card subscription or single donations
- Hire me
- Subscribe at YouTube
- PASS POSTS ON TO OTHERS
Discover more from William M. Briggs
Subscribe to get the latest posts sent to your email.
