Statistics · easy
What is a one-tailed vs two-tailed test, and when is one-tailed defensible?
Asked in Data Analyst interviews, in the Statistics round.
Short answer
Two-tailed asks whether there is any difference in either direction, one-tailed asks about one direction only and puts all of alpha there. One-tailed is defensible only when the other direction is impossible or irrelevant by design, decided before the data.
| One-tailed | Two-tailed | |
|---|---|---|
| Alternative | The effect is in this direction | The effect is in either direction |
| Rejection region | All of alpha in one tail | Alpha split across both tails |
| Power | Higher, for that direction | Lower, but covers both |
| If it moves the other way | You cannot claim anything | You detect it |
| Defensible when | The other direction is decided in advance and would not change the action | The default, and what to use when unsure |
How to answer it
Definition
The difference is where the rejection region sits. A two-tailed test splits alpha across both tails and can detect an effect in either direction. A one-tailed test puts all of alpha in one tail, gaining power there and giving up the ability to detect anything in the other. The table above compares them.
Example
alpha = 0.05, large sample
two-tailed: reject if |z| > 1.96
one-tailed: reject if z > 1.645 (upper tail only)The lower critical value is exactly the extra power, and it is bought by agreeing in advance to ignore the other direction entirely.
When it is defensible
Both conditions have to hold, and they rarely do:
- The direction is specified BEFORE seeing any data, and written down.
- An effect in the other direction would lead to the same action as no effect at all.
A non-inferiority test is the honest example: you want to show a cheaper supplier is not worse, and "meaningfully better" and "the same" both lead to switching, so only one direction matters.
When it is not
Most product experiments. If a new checkout page reduced conversion by three points you would very much want to know, and a one-tailed test in the positive direction cannot tell you. Choosing one-tailed after seeing which way the data went is p-hacking with extra steps: it turns a p of 0.08 into 0.04 by redefining the question.
Key points
- Two-tailed is the default, and the right answer when unsure.
- One-tailed roughly halves the p-value, which is exactly why it is abused.
- The direction must be pre-registered. Deciding afterwards invalidates it.
- Ask what you would do if the effect went the other way. If the answer is "act differently", you need two tails.
- Most guidance and most tooling default to two-tailed for this reason.
FAQ
- Does one-tailed double my power? Not double, but it is a real gain, equivalent to a meaningfully larger sample.
- Can I report both? No. Pick one in advance and report that one.
- What about non-inferiority testing? That is the canonical defensible case, and naming it shows you know when the exception applies.
The 30-second answer
A two-tailed test splits the significance level across both tails, so it can detect an effect in either direction. A one-tailed test puts all of alpha in one tail, which raises power for that direction and gives up the other entirely.
It is defensible only when two things hold: the direction was fixed before any data was seen, and an effect the other way would lead to exactly the same decision as no effect. Non-inferiority testing is the clean example.
For most product experiments I would use two-tailed, because if a change made conversion worse I would absolutely want to know. And choosing one-tailed after seeing which way the data moved is just p-hacking, since it halves the p-value by redefining the question after the fact.
In the interview
- What they are checking: whether you know the choice is made before the data, and that one-tailed halves the p-value for the same result.
- Common mistake: reporting a one-tailed p-value because the two-tailed one was 0.08.