Mixed-Precision Computing
Mixed precision uses lower-precision arithmetic where accuracy allows and higher precision where it is needed, trading exactness for speed and memory.
The trade
Lower-precision formats (FP16, BF16, FP32 versus FP64) use fewer bits, so they move faster through memory, occupy less of it, and often run several times faster on hardware, especially on tensor cores. The cost is reduced accuracy and a smaller representable range. Mixed precision is the discipline of applying low precision only where the numerics tolerate it while keeping critical steps in higher precision.
Iterative refinement
A classic technique is iterative refinement for linear systems: factor the matrix in low precision (fast), then correct the solution with residuals computed in high precision. The bulk of the work runs at the fast precision, while a few high-precision correction steps recover accuracy close to a fully high-precision solve. This pattern generalizes: do the heavy lifting cheaply, then correct in a wider format.
- Low precision: less memory traffic, higher throughput, narrower range.
- High precision: accuracy and range where the algorithm is sensitive.
- Loss scaling and wide accumulation guard against underflow and drift.
- Always validate low-precision results against a high-precision reference.
Where the danger lives
Low precision fails silently in specific spots: summing many small numbers (accumulation should stay wide), subtracting nearly equal numbers (catastrophic cancellation), and values near the format's range limits (overflow to infinity or underflow to zero). Deep-learning training addresses underflow with loss scaling, multiplying gradients up before they are stored in FP16 and dividing back afterward. Scientific codes isolate reductions, norms, and ill-conditioned steps to keep them in FP32 or FP64.
Discipline for simulation
In fusion modeling, mixed precision can accelerate matrix-heavy and machine-learning surrogate steps, but any speedup is only valid if the result matches a high-precision baseline on the quantities that matter. For a Hyperion solver, convergence residuals and conserved quantities are checked in full precision even when inner kernels run reduced; a faster answer that fails conservation is not an answer.