Statistics · medium
What is the difference between parametric and non-parametric tests, and when do you choose each?
Asked in Data Analyst interviews, in the Statistics round.
Short answer
Parametric tests assume a distribution for the data or its errors, like the t-test assuming roughly normal means. Non-parametric tests like Mann Whitney or a permutation test work on ranks or resampling. Choose non-parametric for small skewed samples, ordinal data or heavy outliers.
| Parametric | Non-parametric | |
|---|---|---|
| Assumes | A distribution, usually normal | Very little about shape |
| Examples | T-test, ANOVA, Pearson | Mann-Whitney, Wilcoxon, Kruskal-Wallis, Spearman |
| Tests | Means | Ranks or medians |
| Power when assumptions hold | Higher | Lower |
| With outliers or skew | Misleading | Robust |
| Choose it when | The data is roughly normal or n is large | Small n, heavy skew, ordinal data |
How to answer it
Definition
Parametric tests assume the data follows a distribution with parameters, usually normal, and test statements about those parameters. Non-parametric tests assume very little about shape and generally work on ranks. The table above compares them.
Example
Comparing two groups' revenue
Parametric Welch's t-test on the means
needs the sample MEANS to be roughly normal, which the CLT
usually delivers at a decent n
Non-parametric Mann-Whitney U on the ranks
needs almost nothing about shape
tests whether one group tends to exceed the otherThe tradeoff
When the assumptions hold, the parametric test has more power: it uses the actual values rather than only their order, so it needs a smaller sample for the same detection ability. When the assumptions fail, that extra power is illusory and the p-value is not trustworthy.
They answer different questions
This is the subtlety worth raising. A t-test compares means. Mann-Whitney asks whether a randomly chosen value from one group tends to exceed one from the other. With symmetric distributions those nearly coincide; with badly skewed ones they can disagree, and the mean may not even be the quantity you care about.
Key points
- Parametric assumes a shape and is more powerful when that assumption holds.
- Non-parametric works on ranks, is robust to outliers and skew, and costs some power.
- At a large n the CLT often rescues the parametric test even on non-normal data.
- Small samples, heavy skew or ordinal data point to non-parametric.
- They test different hypotheses, so a disagreement between them is informative rather than a contradiction.
FAQ
- What are the usual pairs? t-test with Mann-Whitney, paired t-test with Wilcoxon signed-rank, ANOVA with Kruskal-Wallis, Pearson with Spearman.
- Should I test for normality first? Preferably not. A normality test on a big sample rejects for trivial deviations. Look at a Q-Q plot and the sample size.
- Is the bootstrap non-parametric? Yes, and it is often the better modern answer, since it works on the statistic you actually care about.
The 30-second answer
Parametric tests assume the data comes from a particular distribution, usually normal, and test its parameters. Non-parametric tests make almost no assumption about shape and generally work on ranks.
The tradeoff is power. When the assumption holds the parametric test needs less data for the same sensitivity, because it uses the values rather than just their order. When it fails, that power is illusory.
I choose based on sample size and shape. At a large n the central limit theorem usually rescues a t-test even on skewed data. With a small sample, heavy skew or ordinal data I would use Mann-Whitney or a bootstrap. And I would note that they answer different questions, so if they disagree that tells me something about the distribution rather than one being wrong.
In the interview
- What they are checking: what "parametric" assumes, two or three named alternatives, and the sample-size point.
- Common mistake: calling Mann Whitney a test of medians. It tests whether one distribution tends to produce larger values than the other.