Statistics · medium
What are the assumptions of linear regression, and how do you check them?
Asked in Data Scientist interviews, in the Statistics round.
Short answer
Linear relationship, independent errors, constant error variance, normal errors for inference, and no severe multicollinearity. Check with residual plots, a Durbin Watson or autocorrelation plot, a scale location plot, a Q-Q plot, and VIF.
How to answer it
Definition
Ordinary least squares gives unbiased, efficient estimates with valid standard errors only under a set of assumptions. Different assumptions protect different things, which is why it matters which one is violated.
The assumptions
Linearity the relationship really is linear in the parameters
Independence observations are independent of each other
Homoscedasticity residual variance is constant across fitted values
Normality the RESIDUALS are roughly normal (matters for inference)
No perfect
multicollinearity predictors are not exact linear combinations
Exogeneity the errors are uncorrelated with the predictorsHow to check them
import statsmodels.api as sm
model = sm.OLS(y, sm.add_constant(X)).fit()
resid, fitted = model.resid, model.fittedvalues
# linearity and homoscedasticity: residuals against fitted
# a curve -> non-linearity
# a funnel -> heteroscedasticity
# normality: a Q-Q plot of the residuals
# independence: Durbin-Watson, or residuals against time
# multicollinearity: VIF per predictor
print(model.summary())The residuals-against-fitted plot is the single highest-value diagnostic: it shows linearity and constant variance at once, and a curve or a funnel is visible immediately.
What each violation costs
- Linearity broken: the model is biased. This is the serious one.
- Independence broken: coefficients are fine, standard errors are far too small, so everything looks significant. Common with time series and clustered data.
- Homoscedasticity broken: coefficients fine, standard errors wrong. Robust standard errors fix it.
- Normality broken: barely matters at a decent sample size, thanks to the CLT.
Key points
- Normality applies to the residuals, not to the predictors or the outcome. This is the most common misunderstanding.
- Independence is the assumption most often violated and least often checked.
- Heteroscedasticity is cheap to fix: use robust standard errors and move on.
- Non-linearity is the expensive one, because the estimates themselves are wrong.
- Plot the residuals. The summary table will not tell you the shape is wrong.
FAQ
- Does the outcome need to be normal? No. Only the residuals, and only for inference at small samples.
- What if residuals are heteroscedastic? Robust standard errors, a transform such as log, or weighted least squares.
- How do I handle clustering? Cluster-robust standard errors, or a mixed model, rather than pretending the rows are independent.
The 30-second answer
Linearity, independence, constant error variance, roughly normal residuals, and no perfect collinearity between predictors.
The first thing I would do is plot residuals against fitted values, because that one chart shows linearity and homoscedasticity at once: a curve means the relationship is not linear, a funnel means the variance is not constant. Then a Q-Q plot for normality and VIF for collinearity.
What I would emphasise is that they are not equally serious. Non-normal residuals barely matter at a decent sample size. Heteroscedasticity leaves the coefficients fine and just needs robust standard errors. Non-independence makes the standard errors far too small, so everything looks significant. Non-linearity is the one that actually biases the estimates.
In the interview
- What they are checking: five assumptions, a check for each, and which ones affect inference versus prediction.
- Common mistake: saying the features must be normally distributed. It is the errors, and only for inference.