Central Limit Theorem

The central limit theorem explains why sums and averages of many independent variables become normally distributed.

The statement

For independent, identically distributed variables with finite mean μ and variance σ², the standardized sample mean (X̄_n − μ)/(σ/√n) converges in distribution to a standard normal as n grows. The average is approximately N(μ, σ²/n) for large n, regardless of the original distribution's shape.

Why it is remarkable

Kronos motion — central column

The individual variables can be skewed, discrete, or bounded; the theorem still delivers a normal limit for their average. This universality is why the normal distribution appears so often — measured quantities are frequently sums of many small independent contributions.

How fast it converges

Convergence speed depends on the underlying shape. Symmetric, light-tailed variables converge in a handful of samples; strongly skewed ones need many more. The rule of thumb 'n ≥ 30' is only a rough guide, unreliable for heavy tails or extreme skew.

python

import random, statistics
def sample_mean(n):
    return sum(random.random() for _ in range(n))/n  # uniform inputs
means = [sample_mean(30) for _ in range(10000)]
print(round(statistics.mean(means),3), round(statistics.pstdev(means),4))

Consequences

The theorem is what makes the sample mean's error scale as σ/√n and justifies normal-based confidence intervals and z-tests. Monte Carlo error bars are a direct application: the estimate is approximately normal around the true value with standard error σ/√n.

Limits

When variance is infinite the classical theorem fails and stable distributions with heavy tails take over. Strong dependence between terms can also break it. Both cases are exactly where naive normal error bars mislead.