Statistics · medium
How do you interpret a regression coefficient, and when is that interpretation wrong?
Asked in Data Analyst interviews, in the Statistics round.
Short answer
A one-unit increase in the feature is associated with a coefficient-sized change in the outcome, holding the other features fixed. Wrong when features cannot move independently, when the relationship is nonlinear, when the outcome is logged or logistic, or when you read it as causal.
How to answer it
Definition
In a linear model, a coefficient is the expected change in the outcome for a one-unit increase in that predictor, holding the other predictors fixed. Every part of that sentence is doing work.
Example
price = 50,000 + 120 * sqft - 3,000 * distance_km
+120 each extra square foot is associated with $120 more,
for houses at the same distance
-3,000 each extra km from the centre is associated with $3,000 less,
for houses of the same sizeWhen the interpretation breaks
- It is associational, not causal. Unless the predictor was randomized, "holding others fixed" is a statistical operation, not an intervention.
- Holding others fixed can be impossible. With age and age squared in the model you cannot change one and not the other, so the individual coefficient means little.
- The scale matters. One unit of a variable measured in millions is a different claim from one measured in ones. Standardise to compare magnitudes.
- Collinearity destabilises it. With correlated predictors the split between them is arbitrary and a sign can flip when another variable is added.
- Controlling for a mediator removes the effect you wanted. If ads drive visits and visits drive sales, controlling for visits hides the effect of ads.
- After a transform the units change. With a log outcome the coefficient is roughly a percentage change, not an absolute one.
- Extrapolating past the observed range is unsupported. The line continues; the evidence does not.
Key points
- "Holding others fixed" is the phrase to say, and the phrase to be suspicious of.
- Association by default. Causation needs randomization or a defended design.
- Standardise coefficients before comparing importance across predictors.
- With a log outcome, multiply by 100 for an approximate percentage reading.
- In logistic regression the coefficient is a log-odds change, and exponentiating gives an odds ratio, not a probability.
FAQ
- Can I rank predictors by coefficient size? Only on the same scale, so standardise first. Even then it is not a causal ranking.
- What does the intercept mean? The predicted outcome when every predictor is zero, which is often meaningless. Centring makes it interpretable.
- Why did my sign flip when I added a variable? Collinearity, or you have just controlled for a confounder or a mediator. Which one changes the story entirely.
The 30-second answer
A coefficient is the expected change in the outcome for a one-unit increase in that predictor, holding the others constant. So in a house-price model, a coefficient of a hundred and twenty on square footage means a hundred and twenty dollars more per square foot, for houses at the same distance from the centre.
Where it goes wrong is mostly that phrase, holding others constant. It is a statistical operation rather than something you can necessarily do in the world, and it is associational unless the predictor was randomized.
The specific traps I would name are controlling for a mediator, which removes the very effect you are measuring, collinearity, which makes the split between two correlated predictors arbitrary and can flip a sign, and extrapolating outside the range the data actually covers.
In the interview
- What they are checking: the conditional clause, units, transformed outcomes, and the causal caveat.
- Common mistake: reading a logistic coefficient as a percentage point change in probability.