Statistician to the Stars!
Here’s a common, classical statistics problem. Uncle Ted’s chain of Kill ’em and Grill ’em Venison Burgers tested two ad campaigns, A and B, and measured the sales of sausage sandwiches for 20 days under both campaigns. This was done, and it was found that mean(A) = 421, and mean(B) = 440. The question is: are the campaigns different?
In Part II of this post, I will ask the following, which is not a trick question: what is the probability that mean(A) < mean(B)? The answer will surprise you. But for right now, I merely want to characterize the sales of sausages under Campaigns A and B. Rule #1 is always look at your data! So we start with some simple plots:
I will explain box and density plots elsewhere; but for short: these pictures show the range and variability of the actual observed sales for the 20 days of the ad campaigns. Both plots show the range and frequency of the sales, but show it in different ways. Even if you don’t understand these plots well, you can see that the sales under the two campaigns was different. Let’s concentrate on Campaign A.
This is where it starts to get hard, because we first need to understand that, in statistics, data is described by probability distributions, which are mathematical formulas that characterize pictures like those above. The most common probability distribution is the normal, the familiar bell-shaped curve.
The classical way to begin is to then assume that the sales, in A (and B too), follow a normal distribution. The plots give us some evidence that this assumption is not terrible—the data is sort of bell-shaped—but not perfectly so. But this slight deviation from the assumptions is not the problem, yet.
The answer to the headline is, unfortunately, yes. The Sunday, 10 February 2008 New York Post reported this sad case of a woman at Mercy Medical Center in New York City. The young woman went to the hospital and had a mammogram, which came back positive, indicating the presence of breast cancer (she also had follow-up tests). Since other members of her family had experienced this awful disease, the young woman opted to have a double mastectomy and to have have implants inserted after this. All of which happened. She died a day after the surgery.
That’s not the worst part. It turns out she didn’t have cancer after all. Her test results had been mixed up with some other poor woman’s. So if she never had the mammogram in the first place, and made a radical decision based on incorrect test results, the woman would not have died. So, yes, having a mammogram can lead to your death. It is no good arguing that this is a rare event—adverse outcomes are not so rare, anyway—because all I was asking was can a mammogram kill you. One case is enough to prove that it can.
But aren’t medical tests, and mammograms in particular, supposed to be error free? What about prostate exams? Or screenings for other cancers? How do you make a decision whether to have these tests? How do you account for the possible error and potential harm resulting from this error?
I hope to answer all these questions in the following article, and to show you how deciding whether to take a medical exam is really no different than deciding which stock broker to pick. Some of what follows is difficult, and there is even some math. My friends, do not be dissuaded from reading. I have tried to make it as easy to follow as possible. These are important, serious decisions you will someday have to make: you should not treat them lightly.
You can download a (non-updated) pdf version of this paper here.
This article will provide you with an introduction and a step-by-step guide of how to make good decisions in particular situations. These techniques are invaluable whether you are an individual or a business.
The results that you’ll read about hold for all manner of examples—from lie detector usefulness, to finding a good stock broker or movie reviewer, to intense statistical modeling, to financial forecasts. But a particularly large area is medical testing, and it is these kinds of tests that I’ll use as examples.
Many people opt for precautionary medical tests—frequently because a television commercial or magazine article scares them into it. What people don’t realize is that these tests have hidden costs. These costs are there because tests are never 100% accurate. So how can you tell when you should take a test?
Under what circumstances is it best for you to receive a medical test? When you “Just want to be safe”? When you feel, “Why not? What’s the harm?”
In fact, none of these are good reasons to undergo a medical test. You should only take a test if you know that it’s going to give accurate results. You want to know that it performs well, that is, that it makes few mistakes, mistakes which could end up costing you emotionally, financially, and even physically.
Let’s illustrate this by taking the example of a healthy woman deciding whether or not to have a mammogram to screen for breast cancer. She read in a magazine that all women over 40 should have this test “Just to be sure.” She has heard lots of stories about breast cancer lately. Testing almost seems like a duty. She doesn’t have any symptoms of breast cancer and is in good health. What should she do?
What can happen when she takes this (or any) medical test? One of four things:
Here is a question I added to my chapter on logic today.
New York City “Health Czar” Thomas Frieden (D), who successfully banned smoking and trans fat in restaurants and who now wants to add salt to the list, said in an issue of Circulation: Cardiovascular Quality and Outcomes that “cardiovascular disease is the leading cause of death in the United States.” Describe why no government or no person, no matter the purity of their hearts, can ever eliminate the leading cause of death.
I’ll answer that in a moment. First, Frieden is engaged in yet another attempt by the government to increase control over your life. Their reasoning goes “You are not smart enough to avoid foods which we claim—without error—are bad for you. Therefore, we shall regulate or ban such foods and save you from making decisions for yourself. There are some choices you should not be allowed to make.”
The New York Sun reports on this in today’s paper (better click on that link fast, because today could be the last day of that paper).
“We’ve done some health education on salt, but the fact is that it’s in food and it’s almost impossible for someone to get it out,” Dr. Frieden said. “Really, this is something that requires an industry-wide response and preferably a national response.”…”Processed and restaurant foods account for 77% of salt consumption, so it is nearly impossible for consumers to greatly reduce their own salt intake,” they wrote. Similarly, regarding sugar, they wrote: “Reversing the increasing intake of sugar is central to limiting calories, but governments have not done enough to address this threat.”
Get that? It’s nearly impossible for “consumers” (they mean people) to regulate their own salt intake. “Consumers” are being duped and controlled by powers greater than themselves, they are being forced to eat more salt than they want. But, lo! There is salvation in building a larger government! If that isn’t a fair interpretation of the authors’ views, then I’ll (again) eat my hat.
The impetus for Frieden’s latest passion is noticing that salt (sodium) is correlated—but not perfectly predictive of, it should be emphasized—with cardiovascular disease, namely high blood pressure (HBP). This correlation makes physical sense, at least. However, because sodium is only correlated with HBP, it means that for some people average salt intake is harmless or even helpful (Samuel Mann, a physician at Cornell, even states this).
What is strange is that, even by Frieden’s own estimate (from the Circulation paper), the rate of hypertension in NYC is four percentage points lower than the rest of the nation! NYC is about 26%, the rest of you are at about 30% If these estimates are accurate, it means New York City residents are doing better than non residents. This would argue that we should mandate non-city companies should emulate the practices of restaurants and food processors that serve the city. It in no way follows that we should burden city businesses with more regulation.
Sanity check:
[E]xecutive vice president of the New York State Restaurant Association, Charles Hunt…said any efforts to limit salt consumption should take place at home, as only about 25% of meals are consumed outside the home.
“I’m concerned in that they have a tendency to try to blame all these health problems on restaurants…This nanny state that has been hinted about, or even partially created, where the government agencies start telling people what they should and shouldn’t eat, when they start telling restaurants they need to take on that role, we think its beyond the purview of government,” Mr. Hunt said.
Amen, Mr Hunt. It just goes to show you why creators and users of statistics have such a bad reputation. Even when the results are dead against you, it is still possible to claim what you want to claim. It’s even worse here, because it isn’t even clear what the results are. By that I mean, the statements made by Frieden and other physicians are much more certain than they should be given the results of his paper. Readers of this blog will not find that unusual.
What follows is a brief but technical description of the Circulation paper (and homework answer). Interested readers can click on.
