Machine Learning ยท medium
How do you handle class imbalance in classification?
Asked in ML Engineer interviews, in the Machine Learning round.
Short answer
Resampling (SMOTE/undersample), class weights, threshold tuning, PR-AUC over accuracy.
How to answer it
Definition
When one class is rare, accuracy stops being informative and most models default to predicting the majority. The fix is a combination of the right metric, the right threshold, and sometimes a change to the data or the loss.
Start with the metric
1% fraud rate. A model that predicts "not fraud" for everyone:
accuracy 99% useless
recall 0% catches nothing
PR-AUC ~0.01 correctly terribleGetting the metric right comes first, because every later decision is judged by it. For a rare positive class use precision, recall, F-beta and PR-AUC.
The levers, roughly in order
1. Threshold tuning free. The model already outputs a probability; the
0.5 cutoff is a default, not a result
2. Class weights class_weight="balanced", or scale_pos_weight in
XGBoost. Reweights the loss, invents no data
3. Undersample majority faster training, discards real data
4. Oversample / SMOTE synthesises minority points. Fit INSIDE the CV fold
5. Different framing anomaly detection when positives are very rare or
not well represented by past examplesExample
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_recall_curve
model = LogisticRegression(class_weight="balanced").fit(X_train, y_train)
probs = model.predict_proba(X_val)[:, 1]
prec, rec, thresh = precision_recall_curve(y_val, probs)
# pick the threshold where recall clears the business requirement,
# then read off what precision that costsKey points
- Threshold tuning is the cheapest and most underrated lever. Try it before touching the data.
- Class weights adjust the loss without inventing observations, so they are usually preferable to oversampling.
- SMOTE must be fitted inside the cross-validation fold. Resampling before splitting leaks synthetic neighbours into validation.
- Never resample the test set. It must reflect the real class balance or the metrics are fiction.
- PR-AUC over ROC-AUC when positives are rare: ROC-AUC looks flatteringly high because true negatives dominate.
FAQ
- Does imbalance always need fixing? No. If the model already separates the classes well at a tuned threshold, leave it alone.
- Where does the threshold come from? The cost of a false positive against a false negative. That is a business input, not a hyperparameter.
- Does SMOTE actually help? Sometimes, and less often than its popularity suggests. Weights plus a tuned threshold is a strong baseline.
The 30-second answer
The first thing I would change is the metric, because accuracy is meaningless when one class is one percent: predicting the majority every time scores ninety-nine percent. I would use precision, recall and PR-AUC instead.
Then the cheapest lever, which is the decision threshold. The model outputs a probability and zero point five is just a default, so I would pick the threshold off the precision-recall curve at whatever recall the business needs.
After that, class weights to reweight the loss without inventing data, and only then resampling. If I use SMOTE it has to be fitted inside the cross-validation fold, because resampling before the split leaks synthetic neighbours into validation. And I would never resample the test set, since it has to reflect the real balance.
In the interview
- What they are checking: whether you start with the metric and the threshold before touching the data.
- Common mistake: SMOTE applied before the train/validation split, which leaks synthetic copies of validation points into training and produces a beautiful, fake score.