I love air pollution. Smells like progress. Good for you. Nothing healthier. Except smoking.
There, now that that’s out of the way, let’s get to the paper “Air pollution-induced missed abortion risk for pregnancies” by Liqiang Zhang and a slew of others in the peer-reviewed journal Nature Sustainability.
Now I know what you’re going to say. But it’s true. There is really is a journal with that asinine name. It has papers like “Domino effect of climate change over two millennia in ancient China’s Hexi Corridor”, “Citizen science and the United Nations Sustainable Development Goals”, and of course “Realizing resilience for decision-making”, which opens with the words “Researchers and decision-makers lack a shared understanding of resilience.” University libraries must be cutting back on their dictionaries.
What? Huh? “Get going already”? Oh, very well. Let’s try to take this thing seriously.
The New York Times did. Took it seriously, I mean. It makes for amusing reading as they try to (a) paint pollution as a killer, while (b) denying a person is being killed. Skip it.
So Zhang got “clinical records of 255,668 pregnant women in Beijing from 2009 to 2017.” Which is fine. Now comes part one of the Epidemiologist Fallacy: they never measured actual “air pollution” exposure on any of these women.
Instead, they “computed the air pollutant exposure level of each pregnant woman on the basis of measurements at the nearest air monitoring stations from her residential and working places.” Latitude and longitude.
Can you say over-certainty?
Well, it might be all right, as long as they took the uncertainty of actual exposure into account in all their models. This would widen all their confidence intervals by, oh, two or three times. Did they? No, sir, they did not.
But did they imply that “air pollution” caused miscarriages on the basis of the too-narrow confidence intervals? Yes, sir, they did.
Previous studies have also indicated that maternal long-term exposure to air pollution may mean a higher likelihood of abortion/miscarriage, stillbirth and birth defects.
We investigated several possible causal mechanisms to explain this linkage…
We thus have the second part required for the Epidemiologist Fallacy.
Now for some fun technical niceties. Monsieur (I’m guessing) Zhang et alia reported their results in terms of what they call “odds ratios”. They were not. It’s a very common mistake to make, which many statisticians do make, to call the parameter inside a model an “odds ratio”.
No. What we want is
Pr(miscarriage | level of pollution, data, model)
Pr(no miscarriage | level of pollution, data, model)
That’s an odds (ratio). The confidence interval—which is equivalent to a p-value—about some parameter in a model associated with pollution says nothing directly about the uncertainty in the observable, here a miscarriage. It is a too-common blunder to confuse the uncertainty in the parameter with the uncertainty in the observable.
What this means is that the stated intervals in their “findings” are not only too narrow because of the Epidemiologist Fallacy, they’re extra-too-narrow because of the confusion of parametric with predictive analysis.
“So what’re you’re trying to say, Briggs, is that pollution is a good thing, right?”
Yes, you caught me. That’s exactly what I meant.
“Har har. You think you’re so smart. At least authors are trying to do something.”
That’s the Don’t Just Do Something, Stand There Fallacy. Doing something harmful can be worse than doing nothing.
“But it’s obvious air pollution causes miscarriages.”
“Yes, it is. It certainly isn’t doing these women any good.”
If it’s obvious, then we didn’t need this study.
“Come on. This study at least put some numbers on the problem.”
Lousy ones; numbers you can’t believe.
“Don’t be such an ass. Everybody does studies like this.”