Statistics · medium
Asked in Data Scientist interviews, in the Statistics round.
Posterior is proportional to prior times likelihood. A 99% accurate test for a 1 in 1,000 condition still gives a positive result that is wrong about 90% of the time, because the false positives from the healthy majority outnumber the true positives.
P(A given B) equals P(B given A) times P(A), divided by P(B). Say it in words: update what you believed before by how well the evidence fits.
Use the medical test, and use counts rather than fractions:
The prior (1 in 1,000) is doing the work. The same test in a clinic where half the patients are sick gives a very different posterior.
The data version of the same idea: a fraud model with 99% precision on the validation set was validated on a 50/50 sample. Deployed on traffic where fraud is 0.1%, most of its alerts are false. That is Bayes, and it is why you ask about base rates before you trust a metric.
prior, sens, fpr = 0.001, 0.99, 0.01
p_pos = prior * sens + (1 - prior) * fpr
posterior = prior * sens / p_pos # about 0.09