Statistics · easy
How do you check whether data is normally distributed, and when does it matter?
Asked in Data Analyst interviews, in the Statistics round.
Short answer
Look first: a histogram and a Q-Q plot show skew and tails better than any test. Shapiro Wilk and similar tests reject everything at large n. It matters for small-sample t-tests and regression inference, and much less for means at large n because of the CLT.
How to answer it
Definition
Two separate questions, and the second is the more important one. Checking is mostly visual; whether it matters depends entirely on what you are about to do.
How to check
import scipy.stats as st
st.probplot(x, plot=plt) # Q-Q plot: the honest first look
x.hist(bins=50) # shape, modes, obvious skew
print(st.shapiro(x)) # formal test, useful only at small n
print(x.skew(), x.kurtosis()) # numbers to go with the pictureThe Q-Q plot is the one to lead with. Points on the diagonal mean normal; curvature at the ends shows skew, and an S-shape shows heavy tails. It tells you HOW it departs, which a test's single p-value never does.
Why formal tests disappoint
Shapiro-Wilk and Kolmogorov-Smirnov test the null that the data is exactly normal. At n = 50 they miss real departures; at n = 100,000 they reject for deviations far too small to affect anything. The p-value ends up tracking your sample size rather than your problem, which is why the plot is the better tool.
When it matters
Barely at all a t-test or a regression at a decent sample size, because
the CLT applies to the sample MEAN, not the raw data
Somewhat small samples, under roughly 30, where the CLT has not
arrived yet
Quite a lot anything about individual values rather than an average:
prediction intervals, control limits, quantile estimates
Not at all tree models, which care about order rather than shapeKey points
- Regression assumes normal RESIDUALS, not normal predictors or outcomes.
- The CLT means the mean is approximately normal even when the data is not, which is why most tests survive skew.
- Formal normality tests are dominated by sample size. Prefer a Q-Q plot.
- Skew matters most for statements about individuals, least for statements about averages.
- A log transform often fixes right skew in revenue-like data, at the cost of changing what the coefficients mean.
FAQ
- My data fails Shapiro-Wilk. Is my t-test invalid? Probably not, at a reasonable sample size. Look at the Q-Q plot and the n.
- What about multimodal data? A mean describes it poorly regardless of any test. Investigate the subpopulations instead.
- Should I always transform? No. It changes the interpretation, and if the mean of the untransformed value is what you care about, keep it.
The 30-second answer
I would look at a Q-Q plot first, with a histogram beside it, because that shows how the data departs from normal rather than just whether it does. Formal tests like Shapiro-Wilk are dominated by sample size: on a big dataset they reject for deviations that make no practical difference.
The more useful half is when it matters. For a t-test or a regression at a reasonable sample size, hardly at all, because the central limit theorem applies to the sample mean rather than the raw values, and regression assumes normal residuals anyway rather than normal inputs.
Where it does matter is anything about individual observations rather than averages: prediction intervals, control limits, quantile estimates. There the skew changes the answer directly.
In the interview
- What they are checking: that you look before you test, and that you know the CLT covers the mean at large n.
- Common mistake: running Shapiro Wilk on a million rows and concluding the t-test is invalid.