Statistics · easy
What is the law of large numbers, and how is it different from the Central Limit Theorem?
Asked in Data Scientist interviews, in the Statistics round.
Short answer
The law of large numbers says the sample mean converges to the true mean as n grows. The CLT says how it gets there: the sampling distribution of the mean becomes normal with spread sigma over root n.
| Law of large numbers | Central Limit Theorem | |
|---|---|---|
| Says | The sample mean converges to the true mean | The sample mean is normally distributed |
| About | Where the estimate goes | The shape and spread around it |
| Gives you | Confidence the estimate is not biased | A standard error, so intervals and tests |
| Needs | A finite mean | Finite variance and enough samples |
| Without it | You could not trust any average | You could not put an interval on one |
How to answer it
Definition
Two different promises about the sample mean. The law of large numbers says where it ends up. The central limit theorem says what its distribution looks like along the way. The table above compares them.
Formula
LLN: sample mean -> mu as n grows
CLT: sample mean ~ Normal(mu, sigma / sqrt(n)) for large nExample
import numpy as np
rng = np.random.default_rng(0)
# LLN: one growing sample, watch the mean settle
for n in [10, 100, 10_000, 1_000_000]:
print(n, round(rng.exponential(50, n).mean(), 2))
# CLT: many samples of fixed size, look at the SHAPE of their means
means = [rng.exponential(50, 200).mean() for _ in range(5000)]Output
10 61.44
100 46.82
10000 50.31
1000000 49.98 <- LLN: converging on 50
means: bell shaped, centred 50, sd ~= 50/sqrt(200) = 3.54 <- CLTWhy both are needed
The LLN alone tells you the estimate is going to the right place, but not how far off it is today. Without the CLT there is no standard error, so no confidence interval and no test. The LLN justifies averaging at all; the CLT is what lets you quantify the uncertainty of the average.
Key points
- LLN is about convergence of a value; CLT is about the shape of a distribution.
- LLN needs a finite mean. CLT needs a finite variance, which is the stronger requirement.
- CLT gives the standard error, sigma over root n, and therefore intervals and p-values.
- Neither rescues dependent observations. Both assume independent draws.
- The square root in the CLT is why precision is expensive: four times the data for half the error.
FAQ
- Which is stronger? The CLT, in the sense that it needs finite variance and delivers much more.
- Is there a distribution where the LLN fails? Yes. The Cauchy has no finite mean, and its sample mean never settles.
- Does the LLN mean a coin evens out? No. That is the gambler's fallacy. The proportion converges; the absolute difference in counts can keep growing.
The 30-second answer
They answer different questions about the same quantity. The law of large numbers says the sample mean converges to the true mean as the sample grows, so averaging is worth doing at all. The central limit theorem says the distribution of that sample mean is approximately normal, centred on the true mean with a standard deviation of sigma over root n.
So one tells me where the estimate lands and the other tells me how far off it is likely to be right now. Without the central limit theorem I would have no standard error, and so no confidence interval and no test.
The requirements differ too: the law of large numbers just needs a finite mean, while the central limit theorem needs finite variance as well.
In the interview
- What they are checking: whether you can separate convergence from the shape of the error, and whether you mention independence.
- Common mistake: saying the CLT makes the data normal. It is the mean that becomes normal, the data stays whatever it was.