A pair of statisticians have calculated that, as of 10 June, the probability that Germany takes the 2012 Euro Cup is 26.8%. Russia, they say, comes in with the next highest probability at 17.5%. Poor Ireland was given only a 0.1% chance to win. (I am rooting for Germany and Poland.)
Magne Aldrin and Anders Løland say that
The probabilities are found by simulating (“playing” on a computer) all remaining matches numerous times. The remaining part of the tournament is “played” 10,000 times and from these simulations we calculate all the teams’ chances of winning the Championship, their group, to reach the second round etc.
In other words, they have some model which takes in information selected by the pair and out spits the probabilities. The natural question many want to ask is: are these the right probabilities? And the answer is: yes, as long as they haven’t made any errors in the calculations or in typing out the results for press.
Now compare the MAAL (authors’ initials) probabilities with those produced by the patented Briggs Soccer Model-O-Matic. This model says that Germany has a 6.3% chance of winning. But it also says that Russia has a 6.3% chance; and so does Ireland. So do each of the 16 teams; indeed, they all have equal probability of winning. The natural question now to ask is: are these the right probabilities? And the answer again is yes.
This sounds nuts. How can the MAAL and the BSMOM probabilities both be right? Shouldn’t there exist, somewhere out there, one final set of probabilities which we can discover, a true set which we can all know? Well, no and yes.
The MAAL probabilities are true assuming the model which produced them is true. The BSMOM probabilities are true assuming the model which produced them is true instead. Which model is truly true? Well, if we knew that there would be no use deriving different models. And the MAAL and the BSMOM aren’t the only models in contention: each bookie has a different one, and probably so do other statisticians.
Consider what is happening here. All probabilities (indeed, all statements of knowledge) are conditional on propositions which are known or assumed to be true. These propositions are what the models are. I mean, any logically different set of propositions, which includes observational propositions (e.g. “Germany won so many games last year”), make up what a model is.
For example, the BSMOM model, i.e. its list of propositions, are this: “There are 16 teams in contention (and here is the list), just one of which can win.” From assuming this model is true we deduce the chance that Germany wins is 6.3%, etc.
The MAAL model’s list of propositions is longer and more complex, but in the end it is just a list of propositions which we assume is, or really is, true and thus let us deduce the chance each team will win. All statistical models are the same: mere lists of propositions we assume are true, or really are true, from which we deduce probabilities of events.
There is still a sense that the MAAL is a better model than the BSMOM, however. We have the feeling that, for any situation, there exists what we can call the Omniscient Model. This is a list of propositions which are true and which lead us to deduce that the chance that (in this situation) Germany wins is either 0% or 100%. Since the propositions which make up the OM are true, the chance that Germany wins is 0% or 100%.
There always exists an Omniscient Model, even for quantum mechanical events; the trouble is that, especially for events on the smallest scale, we rarely know what this model is (and for QM it seems we cannot know). But here is where the sense that the MAAL is better than the BSMOM arises: the closer any model’s propositions are to the propositions in the Omniscient Model’s list, the better that model will be. Surely, our gut tells us, the BSMOM is father from the truth.
Our gut is probably right, but it’s relying on its own model which says roughly, “In my experience, models like the BSMOM rarely produce useful information; while models like the MAAL are better.” We can only confirm our gut after the fact, but measuring how close the probabilities of each model are from the actual outcomes. Those measures become yet another proposition which feeds back to our gut (or into a meta-model in a formal sense) so that when we meet MAAL-like and BSMOM-like models in the future we’ll have an idea which to prefer.
Update Fixed asinine mistake and typo. Thanks Uncle Mike and Stephen D!