Massive Over-Certainties In Medicine: Bad Statistics, Bad Experiments

Massive Over-Certainties In Medicine: Bad Statistics, Bad Experiments

Ready to Follow The Science? Let’s listen to the lamentations of its lecturers and lovers.

The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.

Richard Horton, 2015, Editor in chief The Lancet

We earlier did an article looking into Horton’s comments. It’s more than half.

Evidence-based medicine is actually so corrupt as to be useless or harmful…

It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgement of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor.

Marcia Angell, 2009, Former editor in chief New England Journal of Medicine

And how about this?

The medical profession is being bought by the pharmaceutical industry, not only in terms of the practise of medicine, but also in terms of teaching and research. The academic institutions of this country are allowing themselves to be paid agents of the pharmaceutical industry. I think it’s disgraceful.

Dr Arnold Relman, 2002, Former editor New England Journal of Medicine

And this?

I think we have to call it what it is. It is a corruption of the scientific process…It’s led me and others to increasingly question the idea that the manufacturer of the drug could ever be considered the right people to evaluate its effectiveness and safety.

Fiona Godlee, 2016 Editor, BMJ

My favorite, and maybe yours, too, is this one:

If peer review was a drug it would never get on the market because we have lots of evidence of its adverse effects and don’t have evidence of its benefit.

It’s time to slaughter the sacred cow.

Dr Richard Smith, 2015, Former Editor BMJ

There’s more if you need them at the site that collects them.

You will have noticed none of these specimens are from random bloggers or other dissidents, but from Experts themselves.

Now let’s look at the 2017 BMJ review paper “Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13“. The objective was “To determine the availability of data on overall survival and quality of life benefits of cancer drugs approved in Europe.”

Obviously, we’d like the survival prospects to have increased when taking the drugs. The “quality of life” measures are trickier, both because of attempts to quantify the unquantifiable and because these measures are less impressive.


From 2009 to 2013, the EMA approved the use of 48 cancer drugs for 68 indications. Of these, eight indications (12%) were approved on the basis of a single arm study. At the time of market approval, there was significant prolongation of survival in 24 of the 68 (35%). The magnitude of the benefit on overall survival ranged from 1.0 to 5.8 months (median 2.7 months).

Expensive drugs buy an average 2.7 months extra life. In only 35% of the studies were these “significant”. Meaning had a wee p-value, the worst ever measure of performance. Hint: it should be banned.


This systematic evaluation of oncology approvals by the EMA in 2009-13 shows that most drugs entered the market without evidence of benefit on survival or quality of life. At a minimum of 3.3 years after market entry, there was still no conclusive evidence that these drugs either extended or improved life for most cancer indications. When there were survival gains over existing treatment options or placebo, they were often marginal.

Now let’s look at a Science News piece: “A massive 8-year effort finds that much cancer research can’t be replicated”.

Researchers with the Reproducibility Project: Cancer Biology aimed to replicate 193 experiments from 53 top cancer papers published from 2010 to 2012. But only a quarter of those experiments were able to be reproduced, the team reports in two papers published December 7 in eLife…

What’s more, of the 50 experiments from 23 papers that were reproduced, effect sizes were, on average, 85 percent lower than those reported in the original experiments. Effect sizes indicate how big the effect found in a study is. For example, two studies might find that a certain chemical kills cancer cells, but the chemical kills 30 percent of cells in one experiment and 80 percent of cells in a different experiment. The first experiment has less than half the effect size seen in the second one.

Score that one against p-values, too.

Not only p-values, of course. Experimenter control of original data is also huge. This doesn’t always mean cheating, though it means that at least sometimes. But it’s too easy to push things in desired directions, even unthinkingly, when your hand is on all the levers.

When the experiment meets Reality and that control disappears? Well, we see what we have seen here. These two papers are not unusual. There are scores of them, all with the same story and with the identical conclusion: Be not too certain. Do not blindly trust The Science, for it is more often wrong than right.

Then we recall the past two years and the vexxine mania. Authorities who should know better were touting 95-100% efficacy rates—against infection, forsooth!—based on Pfizer’s initial data. (Regular readers will recall I said it couldn’t be greater than half that in real life.)

We were screamed at “Follow The Science!” That should have meant evincing considerable skepticism and cautious use. And not a moral panicked headline rush headline into the abyss.

The results have been entirely as we on Team Reality predicted.

Similar results will continue to hold, not just in medicine, but in all areas of science in which the practitioners have a strong interest in the outcomes.

Buy my new book and learn to argue against the regime: Everything You Believe Is Wrong.

Subscribe or donate to support this site and its wholly independent host using credit card or PayPal click here; Or go to PayPal directly. For Zelle, use my email.


  1. bruce g charlton

    I can’t speak for the US editors – but the three cited from the British journals (BMJ and Lancet) are all known to me as leaders in the crass leftist politicization of their journals, promotion of peer review above truth, and the elimination of real science from their pages.

    None of the three have the slightest interest in or knowledge of real science – they are just fashion-obsessed ideological propagandists.

    That they affect a concern with the destruction of genuine clinically-focused medical research is sheer hypocrisy – or stark insensibility.

    (But I believe Marcia Angell was sincere in her concern – by her subsequent actions.)

  2. Jim Fedako

    The major health gains were realized a century ago by clean water, closed sewage systems, etc. The pharmaceutical companies piggy-backed on those gains and called them their own. Now we have expensive drugs chasing gains that are less and less marginal and more and more nonexistent, at best — outright dangerous in many cases. So the need to cheat to imply efficacy is real.

  3. Incitadus

    The ever growing ‘war on cancer’ grift is a real money maker. My lurking suspicion
    is that most of the ‘cancer survivors’ treated with these quack drugs never had cancer
    in the first place. That would imply the entire medical profession is in on the grift.
    After two years of the covid op, with an overall survival rate of 99.89%, and the failure
    of rank and file doctors to address the absurdity of the measures taken resulting
    in the destruction of the entire world economy this premise is not easily dismissed.
    The funny thing is that by following the entirely bogus science that contradicted natural
    herd immunity and pushing ineffective and unnecessary vaccines their profession was
    one of the hardest hit financially. It is now viewed with a gimlet eye by an ever increasing
    proportion of the public as profits continue to plummet.

  4. many thanks for the article and for the comments. the peer review system has changed since the 65 years when I was first involved in it.

  5. I have to second Bob Kurland’s comment, though it hasn’t been 65 years yet for my own dealings with peer review. Back when I was introduced to it, people were insistent on questioning your results and having you do as extensive an error analysis and propagation as you could do. That was what was seen as a quality paper, given science is a conditional endeavor in a contingent existence. It has greatly changed over the some 50 years for me.

    It is all political now. We used to scoff at Lysenko. We can see how Lysenko arose and what sin comes from it.

    @ Incitadus,
    While I can’t speak for the current situation; I do know that in the past (I have worked in pathology labs), diagnosis of cancer typically required a biopsy specimen, often obtained by open surgery. Did mistakes happen when examining the specimen? Sure; but it cost you if you made that kind of serious mistake too often.

  6. Nice! one of the best short pieces ever – I’m going to pass it on to people who should, but will not, read it.

  7. Ann Cherry

    This great post by Briggs brings to mind another profitable con perpetrated by the medical//pharmaceutical complex, “The Great Cholesterol Con” which is the title of a very entertaining and informative book by Dr. Malcolm Kendrick. The book’s description says it best:

    “Statins are the so-called wonder drugs widely prescribed to lower blood cholesterol levels and claim to offer unparalleled protection against heart disease. Believed to be completely safe and capable of preventing a whole series of other conditions, they are the most profitable drug in the history of medicine.

    “In this ground-breaking work, GP Malcolm Kendrick exposes the truth behind the hype, revealing: high cholesterol levels don’t cause heart disease; a high-fat diet – saturated or otherwise – does not affect blood cholesterol levels; and, the protection provided by statins is so small as to be not worth bothering about for most men and all women.

    “Statins have many more side affects than has been admitted and their advocates should be treated with skepticism due to their links with the drugs’ manufacturers. Kendrick lambastes a powerful pharmaceutical industry and unquestioning medical profession, who, he claims, perpetuate the madcap concepts of ‘good’ and ‘bad’ cholesterol and cholesterol levels to convince millions of people to spend billions of pounds on statins, thus creating an atmosphere of stress and anxiety – the real cause of fatal heart disease.

    “With clarity and wit, “The Great Cholesterol Con” debunks our assumptions on what constitutes a healthy lifestyle and diet. It is the invaluable guide for anyone who thought there was a miracle cure for heart disease, an appeal to common sense and a controversial and fascinating breakthrough that will set dynamite under the whole area.”

  8. brad tittle

    There is a category that cannot be looked at…

    “people who have the disease, don’t know they have the disease, don’t know that they have not been treated for the disease”.

    How many people don’t die when they don’t get treatment for a disease they don’t know they have.

  9. Also, Journals, universities, regulators and pharma are totally connected.

    In this case, a drug company made, I think, a 40 million dollar grant to a university just before the professor published a paper supporting IVM. In the second half of the film he is on tape admitting that someone else edited his paper for him.

  10. Incitadus

    cdquarles writes: Did mistakes happen when examining the specimen? Sure;
    but it cost you if you made that kind of serious mistake too often.

    I would just like to point out that it didn’t cost you it cost your patient who actually
    pays for the test twice monetarily and medically. (Or do labs give refunds?) I take your
    point and no one in a medical lab wants to generate errors it’s just that industry parameters
    which are steeped in tiny p-values have lead them astray. Cancer cells are present in everyone
    at one point or another throughout the course of their lives. The immune system destroys them
    on a regular basis so finding them in a biopsy is not necessarily an earth shattering development.
    I think the point is that when cancer has progressed to terminal velocity the medical treatments
    consisting of various drug cocktails at best advance life by a only few months. So where has all the
    money poured into the war on cancer gone it seems like an effort in futility at best or another grift
    propped up by tiny p-values.

    What Have We (Not) Learnt from Millions of Scientific Papers with p-values?

  11. You put it all together, Matt.

    Less academic descriptions can be found in my book Butchered by “Healthcare.” And I ascribe the motives as well. See my website for a far more inflammatory description of the whole scene.

    Readers of this blog can download my book free at: If you want to thank me, review it.


  12. No, it cost you, too. It cost you your job and if you were high enough up the chain, you’d get sued. In some jurisdictions, it would often go against you even when you had sufficient evidence on your side. And no, we are not talking a few stray cancer cells, we’re talking millions/billions of them out of the trillions or quadrillions of the cells in your body. After all, an inch sized tumor in the right place could and did cost you your bodily life.

    That said, you are correct that much of the effort, but not all, given some advancements in earlier detection, is an exercise in delaying the inevitable.

  13. Incitadus

    cdquarles…That was a bit hyperbolic on my part the point being the actual survival
    rates for advanced cancer despite the billions of dollars thrown at it is abysmal. I
    didn’t mean to imply that testing was of no value it’s just that at cells in the billions
    current treatment doesn’t offer much more than a few months life extension. That in
    and of itself may be of value to some. If I were over seventy with a stage four diagnosis
    I would pass on the chemo given the debilitating side effects and diminished quality
    of life.

  14. Peter Morris

    So we should basically treat published medical results the same way we treat nameplate power ratings for wind turbines.

  15. Tom Welsh

    I have a small but growing collection of books by doctors (some of them retired, which tends to give them both leisure and greater freedom). Their titles tend to be along the lines of, “How to stop your doctor from killing you” (I think that one is Dr Vernon Coleman). All of them explain that virtually no progress has been made in curing cancer in the past century. Most of the vast sums of money invested have gone to developing fancy drugs and other treatments that are mostly quite useless. The treatments are still “slash, burn, or poison” and the biggest breakthrough has been the definition of survival as “still alive 5 years after diagnosis” – hence the great interest in earlier diagnosis! Meanwhile the “cancer” industry has grown to huge dimensions and provides a good living to tens of thousands of scientists, doctors, pharma workers, etc.

    Personally, I think that cancer is better avoided than cured. Early white explorers who documented the lives of the indigenous people they met all agreed that cancer was unknown until those people began eating “white man’s food” and living like white people. Hunter-gatherers got far more exercise, far more sunshine, purer water, food suitable to humans, plenty of sleep, etc. And they were not harmed by poisons and pollution.

  16. Tom Welsh

    A large part of the trouble is that disinterested science has been overtaken by science for other ends (mainly profit and power).

    The expensive equipment and supplies required by much of modern science cost so much that scientists need sponsorship, and such sponsorship is not given away free.

  17. Justin

    Religions’ claims are akshually more accurate than science to Briggs.


  18. Briggs



Leave a Reply

Your email address will not be published. Required fields are marked *