Machine Learning · hard · Reported at FinTech
Explain how you'd build an uplift/causal model for a marketing campaign.
Asked in Data Scientist interviews, in the Machine Learning round.
Short answer
Model treatment effect (T-learner/uplift trees) using randomized/quasi-experimental data.
How to answer it
Definition
An uplift model predicts the CHANGE in outcome caused by treating someone, not their probability of converting. It answers who to target, which a response model cannot.
Why a response model targets the wrong people
converts if treated converts if not
Sure things yes yes <- waste the discount
Persuadables yes no <- the only ones worth it
Lost causes no no <- waste
Sleeping dogs no yes <- treatment HURTSA response model ranks by "likely to convert", which puts the sure things at the top. Those people were going to convert anyway, so the campaign spends money and changes nothing. Uplift ranks by the difference, which finds the persuadables.
What it needs
Randomized data. Uplift is the difference between two potential outcomes for the same person, and only one is ever observed, so the treated and untreated groups must be comparable. A holdout group is not optional overhead; it is the training data.
The approaches
Two-model (T-learner) fit one model on treated, one on control, subtract
simple, and the two errors compound
Single model with
interaction one model with treatment as a feature and interactions
(S-learner), predict with 1 and 0, subtract
Uplift trees split on the difference in response between arms
directly, rather than on the outcome
Meta-learners X-learner, R-learner: more robust when the arms are
very unequal in sizeEvaluation is different
You cannot use AUC, because the true uplift per person is never observed. Use a Qini or uplift curve: sort by predicted uplift, and plot the cumulative difference in conversion between the arms as you go down the ranking. A good model front-loads the gain.
Key points
- Uplift predicts the effect of treating, not the likelihood of converting.
- It needs randomized treatment, so keep a holdout.
- Targeting the top of a response model usually spends on people who needed no persuasion.
- Sleeping dogs are real: some people churn when you remind them the subscription exists.
- Evaluate with Qini or uplift curves, not classification metrics.
FAQ
- What if I have no randomized data? Use a quasi-experimental design and state the assumption, or run a small randomized campaign first.
- Why is uplift modeling harder? The signal is a difference between two rates, so it is much noisier than either rate and needs more data.
- Does this apply beyond marketing? Yes: retention offers, pricing, any treatment where the cost of treating the wrong person is real.
The 30-second answer
An uplift model predicts the change in outcome caused by treating someone, rather than their probability of converting. That distinction matters, because a response model ranks by likelihood of converting and so puts at the top the people who were going to convert anyway. Discounting them spends money for nothing.
What I actually want is the persuadables, and I would also watch for sleeping dogs, people who respond worse when contacted.
It needs randomized data, because uplift is a difference between two outcomes and only one is ever observed, so a holdout is the training data rather than lost revenue. I would start with a two-model approach or a single model with treatment interactions, and evaluate with a Qini curve, since ordinary classification metrics cannot be computed when the true uplift per person is never seen.
In the interview
- What they are checking: that you distinguish uplift from propensity and insist on randomized data.
- Common mistake: training a response model on everyone who was contacted and calling the score "uplift".