Regression Testing

A regression test freezes a trusted result so that any future change altering it raises an alarm before the change ships.

Guarding Against Backsliding

A regression is a fault reintroduced into code that previously worked. Regression testing captures a known-good output and re-checks it after every change, so an unintended shift is caught immediately rather than discovered months later in a published figure. It is the memory of the test suite.

Golden Files and Baselines

The common pattern stores a trusted reference output, sometimes called a golden file or baseline, and compares new runs against it within a tolerance. When a change legitimately alters the result, the baseline is updated deliberately, with the reason recorded. The discipline is that a baseline never changes silently.

The Tolerance Question

For numerical code, comparison must be within tolerance, not exact, because floating-point results vary across compilers, hardware, and thread counts. Too tight a tolerance produces false alarms on harmless platform differences; too loose a tolerance hides real regressions. Choosing the tolerance is part of designing the test.

When a Regression Test Fires

• A genuine bug was introduced: fix the code.
• The physics model was intentionally improved: update the baseline and record why.
• A platform or library changed the last digits: widen the tolerance or pin the environment.

At Kronos

Frozen physics numbers act as baselines: a design result such as a reported plasma current or gain is tied to a specific code version and input set, and a regression check confirms the current code still reproduces it. If a model change moves the number, the change is deliberate, documented, and re-frozen, never an accident discovered after publication.