Statistics · medium
Asked in Data Analyst interviews, in the Statistics round.
Parametric tests assume a distribution for the data or its errors, like the t-test assuming roughly normal means. Non-parametric tests like Mann Whitney or a permutation test work on ranks or resampling. Choose non-parametric for small skewed samples, ordinal data or heavy outliers.
Parametric tests assume a shape. The t-test assumes the sample mean is roughly normal, ANOVA assumes normal errors with equal variance, linear regression assumes a form for the errors. When the assumptions hold they are the most powerful tests available.
Non-parametric tests drop the shape assumption. The Mann Whitney U test compares ranks between two groups, the Wilcoxon signed-rank test handles paired data, Kruskal Wallis is the rank version of ANOVA, and a permutation test builds the null distribution by shuffling labels. They cost some power when the data really is normal and gain a lot when it is not.
When to reach for non-parametric: small samples of skewed data (revenue per user with a few whales), ordinal outcomes (a 1 to 5 rating), or heavy outliers you do not want to delete. With large samples the CLT makes the t-test on means robust anyway, so the choice matters most when n is small.
One more option that interviewers like: the bootstrap. Resample the data, recompute the statistic, and read the interval off the distribution. No distribution assumed, any statistic, including medians and ratios.