One of the very first things you learn in statistics is that people, by and large, are idiots. Actually, that’s probably one of the very first things you learn in life in general - once you’ve gotten through those pesky early stages of walking, potty-training, and not putting everything you find in your mouth just because you think shiny is the same thing as tasty, at least. Statistics just helps you to understand the exact extent of the problem.
Said problem can be described as follows: people are phenomenally bad at working out how likely or otherwise a given event is. This should come as no surprise. It’s why casinos can afford to give all their employees tuxedos (at least those ones that don’t dress them up as Roman centurions). It’s why people can shrug their shoulders and say, “Well, massive global warming might never even happen!” as they’re driving their SUVs to the burning, tyre-lit, tree-felling party that’s kicking off just down the street (remember: it’s BYOCO2B!). Ten quid says that when they finally track down whichever slack-jawed nudnik told the nervous workers on the Deepwater Horizon “Damn the safety warnings, full oil ahead!”, they’ll claim they were sure the chances of catastrophe were too small to be worth worrying about. We just don’t have the internal wiring to cope with this stuff.
Irrational actions aren’t our only problem here. Plenty of people say staggeringly stupid things based entirely on what their instincts tell them seems suspicious. That’s why pundits who want us to consider them worthy of our attention are willing to argue two female Supreme Court nominations in a row implies the sinister hand of political correctness. It’s how Daniel Pipes, director of the Middle East Forum think tank (who we’ll return to) can use a single Muslim woman winning a French beauty pageant as evidence of positive discrimination (the French being particularly well known for trying to include their Muslim population of course, and beauty pageant judges being famed the world over for being obsessed with inclusion quotas, rather than the relative merits of the breasts of the women arrayed before them).
Like I said, you have to be very careful about calling foul in situations where your gut is telling you you’ve seen something too unlikely to have happened by chance. Because, and this is with all the love in the world, your gut is stupid. It’s nothing personal. My gut is stupid too; a staggering, bewildered drunkard, forever gaping comically at the latest pretty object to cross its feeble attention span. It places what it’s seen above what it’s told, what it finds today over what it finds yesterday, and what makes it feel better over what it secretly suspects to be true. Who would trust that? You might as well trust a sheep to pilot a space shuttle. I mean, it’s not like there’s anything wrong with the sheep, exactly: you’d just rather hire one of those NASA guys with training and star charts and opposable thumbs, y’know?
In our case, the astronaut we’re looking for is the binomial distribution.
Quick question: if I toss a coin three times, what are the chances of getting exactly two heads in a row? Some of you will already know, and some of you will have your own way of calculating it, but one way to answer the question is to write down every single combination of events. So, for three tosses, you can have HHH, HHT, HTH, THH, HTT, THT, TTH, and TTT. The above list demonstrates that there are eight possible combinations. Assuming no-one’s fiddled with the coin to make one side more likely to appear than the other, all eight possibilities are all as likely as each other, so that’s a probability of 1/8 for each one. Since three of them involve exactly two heads, the chance of getting exactly two heads is 1/8 +1/8 +1/8 =3/8 (or, if you prefer, 37.5%). On the other hand, the chance of getting exactly three heads is just 1/8 (12.5%).
Note the two processes involved here. The first is listing all the possible combinations. The second is counting the number of combinations that have the property you are interested in. I could obtain the chance of getting at least one head, for example, by counting the number of combinations with a head in them, getting 7/8. If we added more coin tosses, we would get a greater number of combinations, but the basic idea would always be the same.
Nor is there is anything special about using a coin. We could roll three dice instead, and ask how many sixes we got. Now, so long as we think in terms of sixes and non-sixes (which we weirdo mathematicians like to write as 6’), there are still eight outcomes: 666, 666’, 66’6, 6’66, 66’6’, 6’66’, 6’6’6, 6’6’6’. The difference is that now, those eight outcomes are no longer equally likely: we have less chance of getting three sixes than three non sixes. In fact, the chance of three sixes is 1/6 x 1/6 x 1/6 = 1/216 , and the chance of three non-sixes is 5/6 x5/6 x 5/6 = 125/216.
Clearly, this is a bit more complicated than the coin toss example, but the basics are the same. Ask me what the chance of exactly two sixes is, and I can still tell you. There are three combinations that have exactly two sixes, and each of them has a probability of 1/6 x 1/6 x 5/6 = 5/216 (well, it could be probability 1/6 x 1/6 x 5/6 or 1/6 x 5/6 x 1/6, or 5/6 x 1/6 x 1/6, I guess, depending which of the three dice was a six, but since multiplication works in any order, it doesn‘t make any difference), so the chance of exactly two sixes is 5/216 + 5/216 + 5/216 =15/216 .
These two processes, listing all the possible combinations and calculating how likely each one is to happen, form the foundations of the binomial distribution; a method by which we can calculate the odds of successes and failures over multiple experiments. It’s called “binomial”, by the way, because there are only two possibilities each time around: the coin land on either heads or tails, the dice is either a six or it's not, A Muslim entrant wins her beauty pageant, or she doesn’t (“distribution” is just a way of saying “This can be used to determine how results will be distributed” - who says maths makes no sense?). You need four things for the binomial distribution to work. The first we’ve covered: whatever you’re looking at can only have two results - often referred to as “success” and “failure”. Secondly, each time you gain a success or a failure, you’ve performed a “trial”, and for the binomial distribution you have to specify in advance how many trials you intend to perform (in our coin and dice examples, I decided on three trials). Thirdly, each trial must have the same probability of success as all the others - so no shaving bits off the coin or bribing the judges midway, thank you. Finally, there can’t be any interference between the trials; they can't interfere with each other at all. For example, you can have two people who both have a 50% chance of winning a race, but only one of them can do it; the result of one trial directly effects the other.
Still with me?
Working out the number of possible combinations is actually really easy. For the coin and dice examples above, there were eight. This is because there were two possible outcomes for the first trial, two for the second, and two for the third: 2 x 2 x 2 = 23 = 8. Had there been five trials, there would be 25 = 32 outcomes. By the time you have ten trials, the number of outcomes, 210, is over a thousand (1024, if you want to be specific, and believe me I do). Working out the chances of a given combination is fairly simple, too, even if it might be time-consuming. We’ve seen how to do it already: just write down the chance of failure once for every failure the combination involves, and the chance of success once for every success the combination involves, before multiplying them all together.
The final hurdle to clear is counting the number of unique combinations that get us the result we wanted. This is a little trickier, so I don‘t intend to go into it now, other than to say it has something to do with Pascal’s Triangle, a triangle (obviously) made from numbers. It starts like this:
1 2 1
1 3 3 1
1 4 6 4 1
and each new line is found by adding together consecutive values in the line above. If you can see how that relates to our coin example, then award yourself a biscuit. If not, don't worry, all you need to know is that there's a way to link the two things (you can learn more here, if you're particularly curious/masochistic). If you’re amazed that you can work out probabilities for a series of experiments by drawing out a triangle made of numbers, then join the club. If you don’t, well, never mind I guess, but you’re probably reading the wrong article.
Now properly equipped, let us return to Mr Pipes’ argument. His claims regarding US and British beauty pageants being biased in favour of Muslim contestants are unconvincing and ridiculous, respectively*, but in trying to include France as well, he tips into the realms of statistically-vacant lunacy that this column exists to combat. It’s difficult to work out exactly how many beauty contests go on in France, but since Pipes mentions the Miss Picardy winner, let’s assume there is a contest for each of the 26 French regions, plus Miss France itself. Since Pipe’s list of examples starts in 2005, we can assume we have five years to play around with, which gives us a grand total of (at least) 135 trials. Each trial can succeed or fail, where in this case we define “success” as a Muslim entrant winning. So far, so binomial. Things get a bit more wobbly when we assume a constant probability of success. After all, some pageants presumably take place in areas with higher Muslim populations than do others. This is something we‘re just going to have to swallow; we have to make some assumptions somewhere, or we could never model the real world at all. Let’s assume the chance of a Muslim winner is 3%, the Muslim proportion of the entire French population. We’re also on thin ice assuming the trials are independent, since the twenty-seventh trial - Miss France itself - depends directly on who won the first twenty six regional finals. Again, though, there's not anything we can do about that, other than take our conclusions with a pinch of salt.
Since Pipes is calling foul over a single winner, I assume he thinks the most likely result would be for these 135 trials to return exactly no successes. So what’s the chance of that? Well, there are 2135 combinations (over four thousand trillion trillion trillion), exactly one of which has no successes. The binomial distribution gives us the probability 0.016 (1.6%). Hardly seems overwhelming evidence of an Islamophile conspiracy, does it?
In fairness to Pipes, one can argue that 3% is too high a probability, since the very societal attitudes Pipes is terrified are being consciously combated might make Muslims less likely to sign up to enter such competitions, as indeed may their own cultural taboos. Indeed, several days after he first opened his mouth, he finally got around to making that argument, which he claimed had always been “implicit“ (i.e. it cannot be proven he hadn‘t thought about it). Even for a probability of 0.5%, though, it’s still more likely there will be at least one Muslim winner than it is that the Islamic contingent record a result of nul points.
The point to all of this is that gut feeling is absolutely not enough to sensibly process these things. Even with the right mathematical tools available, these matters are still massively complicated. I can offer some help regarding investigative method, but without a detailed breakdown of the intake for French beauty pageants and some kind of grasp of the cultural issues involved, I’m really just shooting in the dark almost as much as Pipes is. The difference is, I know I’m doing it. Also that my gun works. Oh, and to stretch the metaphor still further, I know that when things are dark it's better not to fire at all, rather than just see what you hit and then ask “Does that mean they deserved it?”
Like all distributions, and statistics in general, the binomial distribution is not some kind of miraculous method by which the inner workings of the universe can be instantly revealed. It is however an excellent process for generating ball-park figures. Perhaps more importantly, it’s a method that forces you to consider exactly what you do and do not know before you start flapping your gums about how those damned foreigners are taking all the best tiaras. Finally, it’s a handy way to remind those who try the latter that if they know there’s a method for checking these suppositions and they deliberately didn’t use it, then they need to shut up. Grown-ups are talking.
*Lamentably, conclusively demolishing Pipes' uniquely nasty brand of passive-aggressive apparent bigotry would involve slightly more mathematics than I'm comfortable including in one column (you people have already worked hard enough), which is the only reason I'm focusing on its weakest part here. For a while I was determined to try, though, purely so I could use the title "Danny Pipes, The Boys Are Calling You (Really Kinda Racist)". Sadly, one cannot really base an entire article around a single pun, but you have to admit: as puns go, that one is pretty good.