Bayesian Inference

Bayesian inference treats unknown parameters as random and updates a prior into a posterior using observed data.

Parameters as random

In the Bayesian view, an unknown parameter θ is described by a probability distribution reflecting our uncertainty about it, not a fixed unknown constant to be estimated once. Data sharpen that distribution.

The update

Kronos motion — data assimilation

Bayes theorem gives the posterior: p(θ | data) ∝ p(data | θ) p(θ). The likelihood weights the prior by how well each parameter value explains the data. The normalizing constant, the evidence, is an integral over θ and is often the hard part to compute.

What the posterior gives you

A point estimate: the posterior mean, median, or mode.
A credible interval: a range containing θ with stated posterior probability.
Full uncertainty: the entire posterior, not just a summary.

Credible versus confidence intervals

A 95% credible interval genuinely contains θ with probability 0.95 given the data and prior — the interpretation people wrongly attach to confidence intervals. This directness is a practical appeal of the Bayesian approach, paid for by the need to specify a prior.

Computation

Only simple models have closed-form posteriors (conjugate priors). In general the posterior is explored numerically, most often by Markov chain Monte Carlo, which draws samples from the posterior without computing the intractable evidence integral. Those samples then approximate any posterior quantity by averaging.

As data accumulate, the likelihood dominates and the posterior becomes insensitive to a reasonable prior — a reassuring convergence that different analysts reach similar conclusions.