Statistician to the Stars!
As is well known by now, a passel of climatologists in the 1970s, including such personalities as Stephen “It’s OK to Exaggerate To Get People To Believe” Schneider, tried to get the world excited about the possibility, and the dire consequences, of global cooling.
From the 1940s to near the end of the 1970s, the global mean temperature did indeed trend downwards. Using this data as a start, and from the argument that any change in climate is bad, and anything that is bad must be somebody’s fault, Schneider and others began to warn that an ice age was imminent, and that it was mainly our fault.
The causes of this global cooling were said to be due to two main things: orbital forcing and an increase in particulate matter—aerosols—in the atmosphere. The orbital forcing—a fancy term meaning changes in the earth’s distance and orientation to the sun, and the consequent alterations in the amount of solar energy we get as a result of these changes—was, as I hope is plain, nobody’s fault, and because of that, it excited very little interest.
But the second cause had some meat behind it; because, do you see, aerosols can be made by people. Drive your car, manufacture oil, smelt some iron, even breath and you are adding aerosols to the atmosphere. Some of these particles, if they diffuse to the right part of the atmosphere, will reflect direct sunshine back into space, depriving us of its beneficial warming effects. Other aerosols will gather water around them and form clouds, which both reflect direct radiation and capture outgoing radiation—clouds both cool and warm, and the overall effect was largely unknown. Aerosols don’t hang around in the air forever. Since they are heavy, over time they will fall or wash out. It’s also hard to do too much to reduce the man-made aerosol burden of the atmosphere; except the obvious and easy things, like install cleaner smoke stacks.
Pause during the 1980s when nothing much happened to the climate.
In Part I of this post, we started with a typical problem: which of two advertising campaigns was “better” in terms of generating more sales. Campaigns A and B were each tested for 20 days, during which time sales data was collected. The mean sales during Campaign A was $421 and the mean sales during Campaign B was $440.
Campaign B looks better on this evidence, doesn’t it? But suppose instead of 20 days, we only ran the campaigns one day each, and that the sales for A was just $421 and that for B was $440. B is still better, but our intuition tells us that the evidence isn’t as strong because the difference might be due to something other than differences in the ad campaigns themselves. One day’s worth of data just isn’t enough to convince us that B is truly better. But is 20 days enough?
Maybe. How can we tell? This is the part that Statistics plays. And it turns out that this is no easy problem. But please stay with me, because failing to understand how to properly answer this question leads to the most common mistake made in statistics. If you routinely use statistical models to make decisions like this—“Which campaign should I go with?”, “Which drug is better?”, “Which product do customers really prefer?”—you’re probably making this mistake too.
In Part I, we started by assuming that the (observable) sales data could be described by probability models. A probability model gives the chance that the data can take any value. For example, we could calculate the probability that the sales in Campaign A was greater than $500. We usually write this using math symbols like this:
Pr(Sales in Campaign A > $500 | e)
Most of that formula should make sense to you, except for the right-hand side of it. The bar at the end, the “|”, is the “given” bar. It means that whatever appears to the right of it is accepted as true. The “e” is whatever evidence we might have, or think is true. We can ignore that part for the moment, because what we really want to know is
Pr(Sales in B > Sales in A | data collected)
But that turns out to be a question that is impossible to answer using classical statistics!
Here’s a common, classical statistics problem. Uncle Ted’s chain of Kill ’em and Grill ’em Venison Burgers tested two ad campaigns, A and B, and measured the sales of sausage sandwiches for 20 days under both campaigns. This was done, and it was found that mean(A) = 421, and mean(B) = 440. The question is: are the campaigns different?
In Part II of this post, I will ask the following, which is not a trick question: what is the probability that mean(A) < mean(B)? The answer will surprise you. But for right now, I merely want to characterize the sales of sausages under Campaigns A and B. Rule #1 is always look at your data! So we start with some simple plots:
I will explain box and density plots elsewhere; but for short: these pictures show the range and variability of the actual observed sales for the 20 days of the ad campaigns. Both plots show the range and frequency of the sales, but show it in different ways. Even if you don’t understand these plots well, you can see that the sales under the two campaigns was different. Let’s concentrate on Campaign A.
This is where it starts to get hard, because we first need to understand that, in statistics, data is described by probability distributions, which are mathematical formulas that characterize pictures like those above. The most common probability distribution is the normal, the familiar bell-shaped curve.
The classical way to begin is to then assume that the sales, in A (and B too), follow a normal distribution. The plots give us some evidence that this assumption is not terrible—the data is sort of bell-shaped—but not perfectly so. But this slight deviation from the assumptions is not the problem, yet.