Computing Library › Numerical Methods
Numerical Methods

Floating-Point Error Analysis

How finite-precision arithmetic introduces rounding error, and how to reason about its accumulation and worst cases.

Numbers a computer cannot represent exactly

Floating-point numbers store a fixed number of significant digits, so most real numbers are rounded to the nearest representable value. The relative error of a single rounding is bounded by machine epsilon, roughly ten to the minus sixteen for standard double precision. Each arithmetic operation on floating-point numbers can introduce one more such rounding, and understanding how these accumulate is the subject of error analysis.

Catastrophic cancellation

Kronos motion — operating point

The most notorious source of error is subtracting two nearly equal numbers. The leading digits cancel, leaving a result dominated by the rounding errors in the original operands: the relative error explodes even though each input was accurate. The classic example is the naive quadratic formula, which cancels badly when the two roots differ in magnitude; rewriting the formula to avoid the subtraction restores accuracy. Recognizing and reformulating cancellation-prone expressions is a core skill.

Accumulation of rounding

Backward error analysis

Rather than tracking the forward error of every operation, the modern approach of Wilkinson asks: of what nearby problem is the computed result the exact answer? If that nearby problem differs from the original only by a tiny amount, the algorithm is backward stable, and the accuracy of the result is then governed by the problem's conditioning. This reframing turns an intractable tangle of individual roundings into a clean statement about the algorithm.

Why it matters in simulation

Large simulations perform enormous numbers of operations, so even tiny per-operation errors can matter, especially in long time integrations and iterative solves near convergence. Sensible scaling, stable formulas, and awareness of cancellation keep computed physics trustworthy; ignoring them can produce plausible-looking but wrong results that are hard to diagnose.