Statistics · medium

Explain statistical power and what increases it.

Asked in Data Scientist interviews, in the Statistics round.

Short answer

Power is the probability of detecting an effect that is really there, one minus the Type II error rate. It rises with sample size, effect size and a higher alpha, and falls with variance. 80% is the usual target.

How to answer it

Power is the probability that the test rejects the null when the alternative is true. If the real lift is 2% and your test has 80% power at that effect, one run in five will miss it.

Four levers, and you should list them in one breath:

• Sample size: more users, more power. Power grows with root n.
• Effect size: bigger real effects are easier to see. You do not control it, but you choose the minimum detectable effect you care about.
• Variance: a noisier metric needs more data. Variance reduction (CUPED, stratification, a less noisy proxy metric) buys power without more traffic.
• Alpha: a looser significance threshold raises power and raises false positives. Rarely the right lever.

The practical use is before the test, not after. Fix alpha at 0.05 and power at 0.8, pick the minimum detectable effect the business would act on, and the formula gives the sample size and therefore the runtime. Computing "observed power" after a null result is circular, and interviewers know it.

from statsmodels.stats.power import NormalIndPower
n = NormalIndPower().solve_power(effect_size=0.05, alpha=0.05, power=0.8)

• What they are checking: the four levers and the fact that power is planned up front.
• Common mistake: running until significance appears. That inflates the false positive rate, it is not power.

Related questions

• When would you use a t-test vs a chi-square test?
• What is Simpson's paradox?
• Explain p-value in plain language.
• How do you determine sample size for an A/B test?
• What is the difference between Type I and Type II error?
• Explain the Central Limit Theorem and why it matters.

Practice this for real

• Check your resume against a Data Scientist job description, free, no account needed.
• Every data interview question, by role and topic
• Scored AI mock interviews, two free, no card