Statistics · medium
What is the difference between correlation and causation, and how do you get closer to causation without an experiment?
Asked in Data Analyst interviews, in the Statistics round.
Short answer
Correlation is co-movement, causation needs a counterfactual. Without a randomized test you approximate one with a natural experiment, difference in differences, regression discontinuity, matching or instrumental variables, each with assumptions you must state.
| Correlation | Causation | |
|---|---|---|
| Claims | These move together | Changing this changes that |
| Evidence | The observed data alone | An intervention, or an assumption you defend |
| Broken by | A confounder, selection, reverse causality | Not applicable once randomized |
| Best evidence | A scatter plot | A randomized experiment |
| Without an experiment | Not enough on its own | Diff-in-diff, instruments, regression discontinuity, matching |
How to answer it
Definition
Correlation is a pattern in observed data: two things move together. Causation is a claim about intervening: change this and that changes. The table above compares them; the useful half of this question is the second part.
Why correlation is not enough
- Confounding. Ice cream sales and drowning both rise with temperature.
- Reverse causality. Does the CRM cause revenue, or do growing companies buy CRMs?
- Selection. Users who opt into a feature were already more engaged.
- Coincidence. Compare enough series and some will track each other beautifully.
Getting closer without an experiment
Difference-in-differences a comparison group that moved in parallel before
the change, so the difference in their trends
after it estimates the effect
Instrumental variables something that shifts treatment but has no other
route to the outcome
Regression discontinuity a cutoff rule where those just above and just
below are otherwise alike
Matching / propensity compare treated units to controls that look the
same on observed covariates
Fixed effects / panel each unit acts as its own control over timeThe honest caveat
Every one of these buys credibility with an assumption, not with data. Matching only balances what you MEASURED, so an unobserved confounder survives it untouched. Difference-in-differences needs parallel trends, which pre-period data can support but never prove. Say which assumption you are leaning on, and how you would probe it.
Key points
- Correlation is a property of the data; causation is a claim about intervention.
- Randomization is the only method that handles unobserved confounders, which is why it is the gold standard.
- Quasi-experimental designs trade an experiment for an assumption. Name the assumption.
- A plausible mechanism is not evidence, but the absence of one should raise suspicion.
- A sensitivity analysis, asking how strong a hidden confounder would have to be, is a strong thing to volunteer.
FAQ
- What about controlling for everything in a regression? It only controls for what you measured, and controlling for a mediator actively removes part of the effect.
- Is a natural experiment as good as a real one? Close, when the assignment really is as-good-as-random. That is the part to defend.
- Do more data help? Not for confounding. A million rows measure the same bias more precisely.
The 30-second answer
Correlation says two things move together in the data I have. Causation says that if I intervene on one, the other changes. The gap between them is usually a confounder, reverse causality, or selection.
Without an experiment I would reach for a quasi-experimental design. Difference-in-differences if I have a comparison group that trended in parallel beforehand, regression discontinuity if there is a cutoff rule, instrumental variables if something shifts treatment with no other route to the outcome, or matching on observed covariates.
The thing I would be honest about is that each of those buys credibility with an assumption rather than with data. Matching only balances what I measured, so an unobserved confounder walks straight through it. I would name the assumption and say how I would test it.
In the interview
- What they are checking: whether you can say why randomization works and what each observational method assumes.
- Common mistake: presenting a regression with control variables as if it settled causation.