Statistics · medium

How do you detect and handle multicollinearity?

Asked in Data Scientist interviews, in the Statistics round.

Short answer

Check VIF/correlation; drop/combine features, use regularization or PCA.

How to answer it

Multicollinearity is predictors that are nearly linear combinations of each other. The model still predicts fine; what breaks is the coefficients. They become unstable, flip sign between samples, and their standard errors inflate, so any story about which feature matters is unreliable.

Detect it with the variance inflation factor: regress each predictor on the others and compute 1 / (1 - R squared). A VIF above 5 is worth a look, above 10 is a problem. A correlation matrix catches pairs but misses a variable that is predicted by three others together, which is why VIF is the tool.

from statsmodels.stats.outliers_influence import variance_inflation_factor
vif = {col: variance_inflation_factor(X.values, i) for i, col in enumerate(X.columns)}

Handling it depends on why you fitted the model:

• If you need interpretable coefficients: drop one of the collinear features, or combine them into one (a sum, a ratio, a PCA component that you can name).
• If you need predictions: ridge regularisation shrinks the correlated coefficients together and stabilises them; tree models are indifferent to it.
• If it comes from dummy variables: you included every level of a category; drop one.

Say clearly that it does not bias predictions. A stakeholder asking "does the model still work?" gets a different answer from one asking "does price matter more than distance?".

• What they are checking: VIF over a correlation matrix, and the distinction between prediction and inference.
• Common mistake: dropping features on correlation alone and losing signal the model needed.

Related questions

• When would you use a t-test vs a chi-square test?
• What is Simpson's paradox?
• Explain p-value in plain language.
• How do you determine sample size for an A/B test?
• What is the difference between Type I and Type II error?
• Explain the Central Limit Theorem and why it matters.

Practice this for real

• Check your resume against a Data Scientist job description — free, no account needed.
• Every data interview question, by role and topic
• Scored AI mock interviews — two free, no card