Normal Distribution
The normal, or Gaussian, distribution is the bell curve that arises whenever many small independent effects add up.
The density
The normal distribution N(μ, σ²) has density f(x) = (1/(σ√(2π))) exp(−(x − μ)² / (2σ²)). It is symmetric about its mean μ, and its spread is set by the standard deviation σ. It is the single most important continuous distribution in statistics.
The 68–95–99.7 rule
About 68% of the mass lies within one σ of the mean, 95% within two, and 99.7% within three. These fractions hold for every normal distribution because all normals are rescaled copies of the standard normal N(0, 1).
Standardization
Any normal variable becomes standard by the z-score Z = (X − μ)/σ. This reduces every normal probability to a lookup against the standard normal CDF, and it is the transformation behind z-tests and standardized residuals.
Why it is everywhere
The central limit theorem explains its ubiquity: sums and averages of many independent contributions tend to a normal regardless of the pieces' own distributions. Measurement noise, aggregated errors, and sample means are approximately normal for this reason.
- Fully specified by two parameters, mean and variance.
- Sums of independent normals are normal.
- Maximum-entropy distribution for a fixed mean and variance.
A caveat
The normal has thin tails, so it badly underestimates the chance of extreme events in heavy-tailed data. Assuming normality where the real distribution has fat tails is a frequent and consequential modeling error.
Two normals combine simply: a linear combination aX + bY of independent normals is again normal, with mean aμ_X + bμ_Y and variance a²σ_X² + b²σ_Y². This closure under addition is rare among distributions and is what makes the normal so convenient in models where many normal quantities are summed or differenced.