Statistics · easy

Explain p-value in plain language.

Asked in Data Scientist interviews, in the Statistics round.

Short answer

Probability of observing data as extreme as this, assuming the null hypothesis is true.

How to answer it

Start with the sentence, then the thing it is not.

The p-value is the probability of seeing a result at least this extreme if there were really no effect. Small p means "this would be surprising under no effect"; that is all it means.

Three things it is not, because the interviewer is listening for at least one of them:

• Not the probability the null hypothesis is true. That needs a prior; it is a Bayesian quantity.
• Not the probability the result is due to chance. It is computed assuming chance is the only thing going on.
• Not the size of the effect. A tiny, useless effect gets a tiny p-value with enough data.

A worked line helps: "We ran the new checkout for 50,000 users each side. Conversion was 3.2% vs 3.0%, p = 0.01. If the two pages really converted the same, we would see a gap this big about 1 time in 100. So we are fairly confident the gap is real; whether 0.2 points is worth shipping is a separate question."

from scipy import stats
# two-proportion z-test, the usual A/B shape
count = [1600, 1500]; nobs = [50_000, 50_000]
from statsmodels.stats.proportion import proportions_ztest
z, p = proportions_ztest(count, nobs)

• What they are checking: that you can state the conditional correctly ("assuming no effect") and separate significance from size.
• Common mistake: "there is a 95% chance the effect is real". Say that and the follow-up questions get harder.

Related questions

• When would you use a t-test vs a chi-square test?
• What is Simpson's paradox?
• How do you determine sample size for an A/B test?
• What is the difference between Type I and Type II error?
• Explain the Central Limit Theorem and why it matters.
• What is p-hacking and how do you avoid it?

Practice this for real

• Check your resume against a Data Scientist job description — free, no account needed.
• Every data interview question, by role and topic
• Scored AI mock interviews — two free, no card