Statistics · easy

What is the difference between Type I and Type II error?

Asked in Data Scientist interviews, in the Statistics round.

Short answer

Type I = false positive (reject true null); Type II = false negative (fail to reject false null).

How to answer it

Type I is a false positive: you reject the null when it is true, and ship a change that does nothing. Its rate is alpha, the significance level you chose. Type II is a false negative: you fail to reject the null when the effect is real, and shelve something that worked. Its rate is beta, and power is 1 minus beta.

The memorable mapping: alpha is how often you cry wolf; beta is how often you miss the wolf.

The point interviewers want next is that the two trade off through sample size. For a fixed n, lowering alpha raises beta. The only way to lower both is more data. So the question "which error is worse here?" is a real design decision:

Related questions

Practice this for real