Statistics

Dawkins On The Double-Blind Experiment

Richard DawkinsThe Edge every year asks top science celebrities a penetrating, yet fun, always fun, question. Last year’s was “What scientific concept would improve everybody’s cognitive toolkit?” One of the correspondent was the pontifical Richard Dawkins. A reader (I apologize, but I cannot re-discover who) asked me to comment on Dawkins’s answer.

I use the word “pontifical” in its ordinary sense, incidentally, but also in honor of Dawkin’s aborted attempt to citizen-arrest Pope Benedict when he landed at Heathrow. Thus endeth our first joke.

Now, I know what you’re going to say. Teasing Dawkins is like plucking low hanging fish from a tree or shooting cherries in a barrel and why don’t I pick on somebody my own size? But it’s Friday after a week of bad news (not my own, the world’s), and I say we need some levity.

Dawkins suggested that it would be keen if everybody carried with them knowledge of the double-blind control experiment. He would that ordinary folk “revel in its elegance.” It is well to point out that the folks in Dawkins’s chosen field are strangers to this most useful probatory device, so one wonders how he happened on the idea. But never mind.

If he had stopped there and had not elaborated, I would have had to concede that even a man as mentally beleaguered as Richard Dawkins occasionally utters a sensible truth. Luckily for us, he pushed on.

He began with what he believed were rhetorical questions:

Why do half of all Americans believe in ghosts, three quarters believe in angels, a third believe in astrology, three quarters believe in Hell? Why do a quarter of all Americans and believe that the President of the United States was born outside the country and is therefore ineligible to be President? Why do more than 40 percent of Americans think the universe began after the domestication of the dog?

He asks that we not blame “stupidity” and instead point to lack of knowledge of the double-blind experiment. Problem is, these phenomena are not subject to the DBE. We might be able to talk somebody out of believing in a ghost by showing her that what caused the picture to move was a stiff breeze, but this will do little to dent her harmless belief in spooks. There just is no double-blind test for where The One was born. Angels and hell follow from religious first principles and if one wants to disprove them, one must answer logical argument with logical argument, not with observation.

What DBE would show the universe began (say) more than 10,000 years ago? There is a small chance that a DBE can prove that astrology is un-predictive, but don’t count on it being persuasive.

Dawkins lists benefits of the DBE:

  1. We would learn not to generalise from anecdotes. It would be bad news for science if we didn’t generalize from anecdotes. Anecdote is the largest generator of scientific inspiration. Experimentation is how we sift the coincidental from the causal.
  2. We would learn how to assess the likelihood that an apparently important effect might have happened by chance alone. In the Big D’s favor, this error is common (this fallacy underlies the frequentist theory of statistics). Things to not happen by “chance.” Chance does not cause anything. Something caused whatever happened to happen. It is only true that we may not know what the cause was, or that we have misidentified the cause.
  3. We would learn how extremely difficult it is to eliminate subjective bias, and that subjective bias does not imply dishonesty or venality of any kind. This lesson goes deeper. It has the salutary effect of undermining respect for authority, and respect for personal opinion. Subjective bias does not always imply venality, but it does sometimes. Scientists prevaricate at rates no different than civilians. But I’m with you when you call for climatologists to use the DBE.

    His second thought is just preposterous. He wants to use the authority of the DBE to undermine respect for authority. Regular readers of Dawkins will recognize that this is his favorite fallacy.

  4. We would learn not to be seduced by homeopaths and other quacks and charlatans, who would consequently be put out of business. Dude. Ain’t gonna happen. The human being loves nothing more than being seduced by charlatans, hence Dawkins’s success.
  5. We would learn critical and sceptical habits of thought more generally, which not only would improve our cognitive toolkit but might save the world. Save the world, forsooth! Never was there a emptier phrase. If you think not, then I challenge you to define exactly what it means. Unless your definition includes words about the Enterprise loosing some photon torpedoes towards a region of space such that a renegade black hole veers off its intended Earth-sucking course, you will have a heavy time of it.

Categories: Statistics

19 replies »

  1. This is like the Verification Principle that could not be verified. Maybe we can extend it to the Precautionary Principle and never use it… just in case 😉

  2. Mr Briggs,

    I suggested that you look at Pinker’s contribution to the Edge question and might like to read the others. I did not expect this. It’s not a great article but it’s a very reasonable argument.

    If you carefully re-read the article you will find that Dawkins blames these implicitly false beliefs on “lack of training in how to think critically, and how to discount personal opinion, prejudice and anecdote, in favour of evidence”, not, as you state, the DBE alone.

    1. I think he means premature and overconfident generalisations from anecdotes.

    2. You misunderstand his colloquial use of “happened by chance alone”.

    3. This paragraph may make more sense to you if you look up James Randi’s investigation into a French scientist who fooled himself about homeopathy (and, I think, got a paper published in nature due to his eminence). The authority of a DBE did indeed outrank the scientist’s authority, as it should.

    4. Surely critical thinking can provide some defenses from such abuses. Try selling Dawkins some of your homeopathic remedies.

    5. Ok, you win on saving the world, but surely critical and sceptical habits do improve our cognitive toolkits.

    Last thing, on what grounds do you call Dawkins a charlatan?

  3. Or the sanity clause.. (don’t he know there ain’t no Sanity Clause?)

    I quite like Dawkins, but he does love to compartmentalise, and in so doing sees the world in the same monochromatic hue as his religious adversaries. Homeopathy might be nonsense, but then again, it might not, and may even contain a grain of truth that will one day reveal something about physiology that is presently unknown. It works for the Queen, and on racehorses, so I’m not ready to dismiss it yet. It’s certainly more plausible than CAGW…

  4. Dr. Briggs,

    I’ve always had a hard time squaring an omnipotent, compassionate God with all of the horrors in the natural world. It seems God can be omnipotent or compassionate but not both.

    It looks like you enjoy beating Dr. Dawkins about the head and shoulders concerning his position on atheism. My assessment is that this is because he fails to preface his statements of belief with “I believe…”, and his assertions of faith with “I have faith that…”. This is sloppy on his part, but nothing a charitable reading cannot overcome.

    Has he otherwise made a fundamental error in reasoning? Or are his conclusions repugnant?

    Or did he key your car after a bar brawl back in ’82?

  5. Eric,

    Oh, I love to pick on Dawkins, I admit it. Anybody who could think up “memes”, who calls religion “child abuse,” and who wants to finger the pope’s collar is a natural target for comedy.

    Yet no mention about proof of God here and I didn’t pull him up for his atheism. And I even allowed that he was right to mention that the DBE is a thing of beauty. Too bad he couldn’t give any useful examples of it.

    I’m glad you asked about fundamental errors in reasoning. His most obvious was his pleasure using authority for “undermining respect for authority.” The worst is made obvious using this anecdote from Stove’s On Enlightenment, quoting from Keynes’s observation of Bertrand Russell:

    Bertie in particular sustained simultaneously a pair of opinions ludicrously incompatible. He held that in fact human affairs were carried on after a most irrational fashion, but that the remedy was quite simple and easy, since all we had to do was to carry them on rationally.

    As Stove comments, “Just two effortless sentences, and yet how fatal they are to any belief in Russell’s political wisdom, or even sense! They are like a bayonet thrust through the heart and out the back.”

    Substitute Dawkins for Russell and you have the same thing: all we need is critical thinking and we can save the world!

  6. Everyone knows photon torpedos would be harmlessly absorbed by a black hole. Nah, this misson calls for treknobabble involving the main deflector and probably the warp coils. Tacyons also will be a major plot device I’d suspect.

  7. Briggs, thanks for the post. I always enjoy seeing someone take Dawkins to task, as he has been the source of an immense volume of nonsense for years.

    However, with respect to #2, there is still some value in the concept of chance. Yes, yes, I know. Physics, and knowing the prior conditions and such can (arguably) mean that there is no real thing as chance; arguably all is determined . . . if only we could identify the prior conditions. I think there are some good reasons for disputing this idea, but even if we concede that there is no chance in the ultimate, interaction of particles sense, the concept of “chance” is still meaningful in many walks of life, even in scientific endeavors.

    Obviously this thread isn’t the place for a lengthy debate on whether there is such a thing as chance or how we might define it, but I think in colloquial terms I have to agree with Dawkins’ #2.

    —–

    John Morris: “Everyone knows photon torpedos would be harmlessly absorbed by a black hole.”

    LOL!

  8. His point 2 sounds like the usual creationist argument, which I doubt is what he is trying to say. What does he mean by “happened by chance alone”?

    I think the response to that point is off the mark, coming from a statistician. “chance” is just an informal word for probability. True, probability is not a direct cause, but probability can explain when causes meet and things happen as a result.

  9. The French scientist I mentioned above was Jacques Benveniste. Many online sources discuss his paper in Nature, James Randi’s investigation, and how a good DBE destroyed his career.

  10. Dawkins chooses some annoying examples, but his general description of the problem holds some weight. People are indeed gullible. We are scarcely removed from witch-hunting and believe entirely too easily in popular myths. My field, forest science, has gone backwards over the last 40 years, and now teaches complete crap, pre-Darwinian nonsense, in our finest universities.

    For instance, the recent literature is rife with the effects of global warming, such as beetle epidemics and forest fires. Yet NA winters and summers have been cooling over the last 15 years. How can a non-existent cause yield an effect? What is taught is that ecosystems are fragile webs subject to collapse under the slightest disturbance. What is real is that nature is hugely resilient and maleable.

    Forests are big, too big to put into test tubes, too big to experimentally control confounding factors. Hence Dawkins’ double-blind experiments, or even no-blind experiments, are impossible to perform. History offers us many deductive clues, but our knowledge of landscape history is paltry and weak. As a consequence, logic and rationality are jettisoned with abandon, replaced by superstition and dreamy deterministic models.

    The result of all this medievalism is massive disaster, destruction, and deprivation. Maybe people are too stupid, too easily seduced, too irrational, unteachable and irredeemable — maybe we are slip-sliding back into the Dark Ages. I hold out some hope, however. This blog, for instance, defies the current.

  11. I always chuckle when I see someone discussing the probability of some or other event which has already happened, completely ignoring the fact that its probability is, by definition, unity.

  12. I agree that double blind experimentation is not the panacea to end all ignorance. But it could be a start if it were applied to ther global warming fiasco. The late Michael Crichton advocated something like it, and he knew a thing or two about it in the medical arena. The idea has a variety of morphs; Will Happer has proposed a “Team B” approach in which an independent group of scienists look att he same data as the “team” and issue their interpretation. Evidently this is done routinely in the defense business.

  13. In the case of anti-depressant medication, using the double blind technique has overstated the efficacy of the drugs.
    Irving Kirsch has argued that drug side effects could account for the difference with the placebo group, a difference by the way that is not very large.

    So, imagine that you are a participant in a trial testing a promising new drug. Following the rules of the ethics committee you are briefed on the possible side effects of the drug and you also hear about the design of the experiment. People are pesky experimental subjects, the first thing they do is try and figure out in which group they have been placed. Experts call this “guessing blind” and humans are rather good at it.

    Substituting the inactive placebo with an active one (with side effects) seems like a good idea, but will it pass the ethics committee?

  14. Like proclaiming sex to be rubbish I rather think that calling Dawkins “mentally beleaguered” says more about the author than it does about his subject.

  15. Outlier: “His point 2 [chance] sounds like the usual creationist argument . . .”

    As opposed to what? The idea that everything is absolutely deterministic from first principles? I think Dawkins is using “chance” in the normal, colloquial sense; and in that sense his point is perfectly reasonable, whether applied to evolution, climate, or science in general.

    —–

    Peter317, I’ve rarely seen anyone try to argue in a vaccuum about the probability of a specific event happening that has already happened. What I have seen is people discuss the probability of an event happening in a particular way or under a particular set of conditions, as support for the thesis that it likely did or didn’t happen that way. The first is, as you point out, nonsensical, but no-one is really questioning whether the event took place. The latter is perfectly valid and is a very common, general way of reasoning about cause and effect in the world.

  16. “1.We would learn not to generalise from anecdotes. It would be bad news for science if we didn’t generalize from anecdotes. Anecdote is the largest generator of scientific inspiration. Experimentation is how we sift the coincidental from the causal.”

    I think what Dawkins means to decry is that anecdotal reasoning persists *after* scientific experimentation has placed it in doubt. Almost every smoker seems to know of a reprehensible relative who smoked 90 a day and was shot at the age of 90 by a jealous husband. The existence of said reprobate is held to insulate the heavy smoker from the medical risks of smoking. Of course if this was what he meant it would have helped to say so.

    ” 2.We would learn how to assess the likelihood that an apparently important effect might have happened by chance alone. In the Big D’s favor, this error is common (this fallacy underlies the frequentist theory of statistics). Things to not happen by “chance.” Chance does not cause anything. Something caused whatever happened to happen. It is only true that we may not know what the cause was, or that we have misidentified the cause.”

    Again what I think he meant was that an observed event, say a spate of particularly nasty tornadoes, may be the a symptom of the natural variability in the weather system rather than a consequence of a particular act or omission. i.e performing a rain dance or not nailing up a lucky horse shoe might not be the cause of the sudden onset of “Twister hell”.

    ” 3.We would learn how extremely difficult it is to eliminate subjective bias, and that subjective bias does not imply dishonesty or venality of any kind. This lesson goes deeper. It has the salutary effect of undermining respect for authority, and respect for personal opinion.

    Subjective bias does not always imply venality, but it does sometimes. Scientists prevaricate at rates no different than civilians. But I’m with you when you call for climatologists to use the DBE.”

    I’m not quite sure how we dose one half of the planet with a GHG and administer a placebo to the other half *and* conceal which half is which from the participating “scientists”.

    ” His second thought is just preposterous. He wants to use the authority of the DBE to undermine respect for authority. Regular readers of Dawkins will recognize that this is his favorite fallacy.”

    The DBE is as subject to abuse as any other methos as Dr Ioannidis has somewhat disturbingly proved : http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ Seems odd to want to reduce appeal to authority, bit replace it with such a graven image

    “4.We would learn not to be seduced by homeopaths and other quacks and charlatans, who would consequently be put out of business. Dude. Ain’t gonna happen. The human being loves nothing more than being seduced by charlatans, hence Dawkins’s success.”

    Dawkins isn’t immune, someone borrowed the devils clothes and flogged him an Anthropic Principle

    ” 5.We would learn critical and sceptical habits of thought more generally, which not only would improve our cognitive toolkit but might save the world. ”

    OK he’s a celebrity and therefore has a couple of get out of jail free hyperbole cards he’s allowed to play. If we could succesfully enhance public education so that everyone had sufficient stats, maths, science and reading comprehension skills to at least distinguish shit from Shinola, and arse from elbow, there would doubtless be a great improvement in society and governance. I’ll tell HMG to get straight on it once they’ve fixed the fact that 17% of UK school leavers can’t read and write properly.

    “Save the world, forsooth! Never was there a emptier phrase. If you think not, then I challenge you to define exactly what it means.”

    Dr Gleick seems to think he knows what it means.

    “Unless your definition includes words about the Enterprise loosing some photon torpedoes towards a region of space such that a renegade black hole veers off its intended Earth-sucking course, you will have a heavy time of it. ”

    That would be ‘class c probes’ programmed to emit a tachyon pulse as they approach the event horizon

  17. Human decency is not derived from religion. It precedes it. I’m so fed up with religious people saying that morals come from religion!

  18. Sam, I know how you feel. I’m so fed up with anti-religious people saying that religion drives people to do amoral things. After all, principles of human decency don’t come from religion, right?

    BTW, just curious. you wouldn’t happen to be *that* Sam Harris, would you?

Leave a Reply

Your email address will not be published. Required fields are marked *