Statistics · easy

How do you check whether data is normally distributed, and when does it matter?

Asked in Data Analyst interviews, in the Statistics round.

Short answer

Look first: a histogram and a Q-Q plot show skew and tails better than any test. Shapiro Wilk and similar tests reject everything at large n. It matters for small-sample t-tests and regression inference, and much less for means at large n because of the CLT.

How to answer it

Start with pictures, because the tests are misleading at exactly the sample sizes you have.

A histogram shows skew and multiple modes. A Q-Q plot puts your sample's quantiles against a normal's, and a straight line means normal, a curve at the ends means heavy or light tails. That is usually the whole answer for a real metric: revenue is right-skewed, latency has a long tail, ratings are bounded.

The formal tests, Shapiro Wilk, Kolmogorov Smirnov, Anderson Darling, answer "is this exactly normal", and with fifty thousand rows the answer is always no, because nothing real is exactly normal. With twenty rows they cannot tell.

When it matters: a t-test on a small sample, confidence intervals from a formula, regression p-values with few rows, and any method that uses the mean and standard deviation to describe a skewed thing. When it does not: the mean of a large sample, which is approximately normal by the CLT whatever the data looks like, and tree-based models, which do not care about distribution at all.

If it matters and the data is not normal: log transform a right-skewed metric, use a rank-based or permutation test, or bootstrap.

Related questions

Practice this for real