Type I and Type II Errors

Every hypothesis test can err two ways: a false alarm or a missed detection, and reducing one tends to raise the other.

The two errors

A Type I error rejects a true null hypothesis — a false positive, with probability α. A Type II error fails to reject a false null — a false negative, with probability β. The power of a test is 1 − β, the probability of correctly detecting a real effect.

The tradeoff

Lowering α to avoid false alarms makes the rejection threshold stricter, which raises β and lowers power. The only way to reduce both errors at once is to gather more data, which sharpens the sampling distribution and separates the hypotheses more cleanly.

Power and sample size

Power increases with the true effect size, with the sample size, and with a larger α. A power analysis done before an experiment chooses n so that a meaningful effect would be detected with acceptable probability, typically 80% or 90% power. Underpowered studies waste effort and produce unreliable positives.

Choosing the balance

Which error is worse depends on consequences. For a safety interlock, a missed detection (Type II) may be far costlier than a false alarm (Type I), arguing for high power even while tolerating more false positives. The thresholds should reflect the real costs of each mistake, not a default convention.

This asymmetry is why blanket use of α = 0.05 with no attention to power is poor practice in high-stakes settings.