Numerical Error

Finite precision introduces rounding, cancellation, and accumulation errors that careful numerics must control.

Absolute and relative error

Absolute error is the difference between a computed value and the true value; relative error divides that by the true value. Floating point bounds relative error per operation, so relative error is usually the meaningful measure.

Rounding error

Kronos motion — control room

Every inexact operation rounds its result to the nearest representable number, introducing an error up to half a unit in the last place. Individually tiny, these errors can build up over millions of operations.

Catastrophic cancellation

Subtracting two nearly equal numbers cancels their leading digits and promotes rounding noise into the significant digits. Reformulating the expression — for example, rationalizing a difference of square roots — often avoids the loss.

Error accumulation

Summing many numbers naively lets rounding errors add up. Compensated summation, such as Kahan's algorithm, tracks the lost low-order bits and adds them back, keeping the running total accurate.

Conditioning and stability

A problem is ill-conditioned if small input changes cause large output changes; an algorithm is unstable if it amplifies rounding error. Reliable results need both a well-conditioned problem and a stable method.

In practice

Simulation of fusion plasmas, like the time-stepped Hyperion models, uses double precision and stability-aware schemes so numerical error stays orders of magnitude below physical modeling uncertainty.

python

def kahan_sum(xs):
    total = 0.0; c = 0.0
    for x in xs:
        y = x - c
        t = total + y
        c = (t - total) - y
        total = t
    return total