Sampling Distributions

A sampling distribution is the distribution of a statistic across repeated samples, and it is what makes inference possible.

The idea

A statistic — a sample mean, variance, or proportion — is itself random, because it depends on which sample you happened to draw. The sampling distribution is the distribution of that statistic over all possible samples of a given size. Inference is reasoning from one observed value back to this distribution.

The sample mean

For n independent draws with mean μ and variance σ², the sample mean has mean μ and variance σ²/n. Its standard deviation, called the standard error, is σ/√n. By the central limit theorem the sample mean is approximately normal for large n.

Standard error versus standard deviation

The standard deviation describes the spread of individual observations; the standard error describes the spread of a statistic. Confusing the two overstates or understates precision. The standard error shrinks with sample size; the standard deviation of the population does not.

Other statistics

The sample variance has a scaled chi-squared sampling distribution under normality; a sample proportion is approximately normal with standard error √(p(1−p)/n). Each statistic has its own sampling distribution, and knowing it is the prerequisite for any confidence interval or test.

When theory is hard

For complicated statistics with no clean formula, the bootstrap approximates the sampling distribution by resampling the observed data with replacement. It turns a difficult analytic problem into a straightforward computation, at the cost of many resamples.