Loss Function
The scalar objective a model minimizes, quantifying how far predictions are from targets.
Definition
A loss function (or objective) maps a model's predictions and the true targets to a single number measuring error. Training is the process of adjusting parameters to make this number small.
Losses can also encode constraints and preferences beyond accuracy, such as fairness penalties, sparsity terms, or physics residuals in a physics-informed model. Because the optimizer will exploit any loophole in the objective, writing a loss is really an exercise in stating precisely what you want.
A subtle risk is that the loss and the true objective diverge: a model minimizes exactly what is written, so any mismatch between the loss and the real goal is silently exploited. This is why proxy losses are chosen with care and why deployed systems are monitored against the outcome that actually matters, not only against the training loss that stood in for it.
Common losses
- Mean squared error: regression, penalizes large errors heavily.
- Mean absolute error: regression, robust to outliers.
- Cross-entropy: classification, compares predicted and true distributions.
- Hinge loss: margin-based classification, used by SVMs.
Design considerations
The loss encodes what you care about. A regression loss that squares errors treats a few large mistakes as very costly; an absolute loss is more forgiving. Regularization terms are often added to the loss to discourage overly complex models.
Why it matters
The loss is the definition of success handed to the optimizer. A poorly chosen loss produces a model that optimizes the wrong thing, even if training appears to converge.
Fusion connection
Kronos surrogate losses weight the quantities engineers most need to predict accurately, such as fusion power and confinement, so the fitted model is trustworthy where it matters for Hyperion.