Skip to content

Simpson's Paradox Explained: When Data Tells Two Opposite Stories

TL;DR: Simpson's paradox is a statistical reversal: a trend that appears in several separate groups disappears or flips when the groups are combined. It is a favourite example in psychology and statistics because it shows how easily our causal intuitions about data go wrong. The most famous case is the 1973 University of California, Berkeley admissions data, where the totals suggested bias against women and the department-level figures did not. Below: the Berkeley case, a kidney-stone example, a baseball example, and how to avoid being fooled.

What is Simpson's paradox?

Simpson's paradox is the name for a situation in which a comparison runs one way inside every subgroup and the opposite way in the pooled total. Group A beats group B in each category, yet B beats A overall.

It sounds like a trick, but it is plain arithmetic. It happens when the groups being compared are not spread evenly across the categories, so that the mix, not the underlying difference, drives the total. The statistician's word for the hidden factor is a confounder or lurking variable.

The name is a little unfair to the history. According to Wikipedia's account, Karl Pearson (1899) and Udny Yule (1903) described similar effects earlier. Edward H. Simpson wrote about the pattern in his 1951 paper "The Interpretation of Interaction in Contingency Tables" in the Journal of the Royal Statistical Society, Series B. The label "Simpson's paradox" came later, from Colin Blyth's 1972 article in the Journal of the American Statistical Association, "On Simpson's Paradox and the Sure-Thing Principle".

The Berkeley admissions case

The best-known real example comes from Berkeley's graduate admissions in autumn 1973.

The campus-wide figures looked bad. According to the data summarised on Wikipedia, there were 12,763 applicants. Of 8,442 men, about 44% were admitted; of 4,321 women, about 35%. A gap of nine percentage points, on those numbers, looked like evidence of discrimination.

Statisticians Peter Bickel, Eugene Hammel and J. W. O'Connell then examined the data department by department. They published their analysis in Science in 1975 ("Sex Bias in Graduate Admissions: Data from Berkeley", vol. 187, pp. 398-404). Their finding was the opposite of what the totals implied. Across the 85 departments, only a handful showed a statistically significant departure from fair admissions: four were biased against women and six against men, according to the Wikipedia summary of the paper. When the numbers were pooled in a way that accounted for where people had applied, the authors reported a small bias in favour of women.

What had happened? Women had applied more often to the most competitive departments, which turned away many applicants of both sexes. The mix of applications, not the behaviour of admissions committees, produced the aggregate gap. In the authors' words, the bias in the aggregated data did not stem from "any pattern of discrimination on the part of admissions committees".

Two cautions are worth adding. First, the authors did not conclude that nothing unfair was going on in the world. They attributed the aggregate disparity to earlier educational screening and socialisation, which shaped the choices women had before they ever applied. Simpson's paradox tells you where not to look for a cause. It does not tell you there is no problem. Second, this is one university in one year.

A medical example: kidney stones

In 1986 a team led by C. R. Charig published a comparison of ways to treat kidney stones in the British Medical Journal: open surgery, percutaneous nephrolithotomy (PCNL), and shockwave lithotripsy. Success meant that the stone was cleared or reduced to under 2 mm after three months. Overall, open surgery succeeded in 273 of 350 patients (78%), and PCNL in 289 of 350 (83%). On the face of it, PCNL wins.

The breakdown by stone size, as it is usually quoted in discussions of the paradox (it is reproduced on Wikipedia, which credits a later analysis by Julious and Mullee in 1994), reverses that:

Stone size Open surgery PCNL
Small stones 93% (81 of 87) 87% (234 of 270)
Large stones 73% (192 of 263) 69% (55 of 80)
All stones 78% (273 of 350) 83% (289 of 350)

Open surgery did better for small stones and better for large stones, yet worse overall. The reason is that open surgery was used far more often on the large, hard-to-treat stones, which pulled its overall success rate down. The confounder was stone size. Because large stones are harder to treat whichever method you use, the subgroup comparison is the fairer one here.

A sporting example: Jeter and Justice

Baseball supplies a compact version, described on Wikipedia and attributed there to the statistician Ken Ross. In 1995, Derek Jeter hit 12 for 48 (.250) and David Justice hit 104 for 411 (.253). In 1996, Jeter hit 183 for 582 (.314) and Justice hit 45 for 140 (.321). Justice had the higher average in both seasons. Combined, Jeter hit 195 for 630 (.310) and Justice hit 149 for 551 (.270), so Jeter came out ahead. Jeter had most of his at-bats in the season when averages were high, and Justice had most of his in the season when his average was low. The weighting flips the total.

Why it matters for psychology

Simpson's paradox is not a cognitive bias. It is a feature of data. But it sits in a cluster of psychology topics because of how people react to it.

We read causes into totals. Faced with a table, most of us leap to a story: women are being rejected, the new treatment works. The paradox shows that the same numbers can support opposite stories, and the choice between them is a causal judgement, not a calculation.

There is no purely mechanical fix. It is tempting to say "always split the data into subgroups". The computer scientist Judea Pearl has argued that this is wrong. As Wikipedia summarises his work, partitioned data reflect the true causal relationship only if the partitioning variables meet what he calls the back-door criterion: they must block spurious paths between the cause and the effect, and must not themselves be affected by the cause. Two different causal diagrams can produce identical numbers while calling for different answers, so the data alone cannot always settle which table to believe. You need to know how the numbers were generated.

It spoils simple league tables. Any ranking of hospitals, schools, teams or products that pools very different groups can be distorted by mix.

How Simpson's paradox shows up in everyday life

  • Comparing averages between groups. Average pay, test scores or recovery rates can reverse when you adjust for age, experience or case severity.
  • Product and website tests. If one version of a page gets more of its traffic from a high-converting source, it can look better overall than it is.
  • News statistics. A headline comparing two countries or two years may be a mix effect in disguise.

The test to apply is simple: before trusting a total, ask what kind of cases ended up in each group, and whether anything about those cases could affect the outcome independently of the thing you are comparing.

What do replication and criticism say?

Most of the effects in a psychology post can fail to replicate. This one cannot, in the usual sense, because it is not an experimental finding. The arithmetic holds every time, and you can check the kidney-stone and baseball numbers above with a calculator.

The real debate is about interpretation:

  • Which table is right? Pearl's point is that the answer depends on causal assumptions, not on the numbers.
  • What does Berkeley show? It is a standard teaching example and is usually presented as "apparent bias that vanishes on closer inspection". The original paper is more careful than that summary: it reports a small pooled bias in favour of women and puts the aggregate gap down to earlier screening and choices, which is a different thing from saying nothing was wrong.
  • Is every reversal a paradox? Not every aggregate-versus-subgroup difference is a puzzle. Whenever the groups truly differ in size and mix, a reversal is the expected result.

Frequently Asked Questions

What is Simpson's paradox in simple terms?

A trend that holds inside every subgroup reverses when you add the groups together. It happens because the subgroups differ in size and make-up, so the mix changes the overall result.

Who discovered Simpson's paradox?

Edward H. Simpson described it in a 1951 paper, but earlier writers, including Karl Pearson (1899) and Udny Yule (1903), had noted similar effects. Colin Blyth gave it the name "Simpson's paradox" in 1972.

What is the Berkeley admissions example?

In autumn 1973, Berkeley's overall figures showed a lower admission rate for women (about 35%) than men (about 44%). Department by department, the pattern largely disappeared, because women had applied more often to highly competitive departments. Bickel, Hammel and O'Connell reported this in Science in 1975.

Is Simpson's paradox a cognitive bias?

No. It is a property of data. The bias is in how we interpret it: we tend to read a causal story from whichever table we are shown first.

How do you avoid Simpson's paradox?

Ask what confounding variables could affect the outcome, check whether the groups differ in mix, and decide on causal grounds whether to trust the pooled or the split figures. Splitting the data is not always the right answer.

Has Simpson's paradox failed to replicate?

It cannot fail to replicate, because it is arithmetic rather than an experimental result. What is open to debate is how to interpret each real-world example.

Learning more

Chunks turns psychology's famous cases into short narrated stories, with a free tier and no ads. The psychology experiments pack is a good place to start.

Andy Shephard, Founder of Chunks

Andy Shephard

Founder of Chunks Microlearning. Software engineer with 15 years of experience.

Start learning today

In just minutes, you can uncover something new and fascinating — with content tailored to spark your curiosity and match your interests.