Overfitting
When a model fits the training data's noise and fails to generalize to new data.
Definition
Overfitting occurs when a model captures noise and quirks specific to the training set rather than the underlying pattern. It shows as low training error but high error on held-out data.
Detecting overfitting requires holding data genuinely separate: information from the test set must never leak into training or model selection, or the reported performance is inflated. Subtle leakage, through preprocessing fit on all data or through repeated tuning on the same test set, is a common and hard-to-spot error.
The amount of data needed to avoid overfitting grows with model complexity, so the same flexible model that overfits a small dataset may generalize well given a large one. This links overfitting directly to data volume and to regularization strength, and it explains why data augmentation, which synthesizes plausible new examples, is often the most effective single remedy in practice.
The opposite failure is underfitting: the model is too simple to capture the real structure, giving high error everywhere.
Symptoms and remedies
- Large gap between training and validation performance.
- Remedy with more data or data augmentation.
- Add regularization to penalize complexity.
- Use cross-validation to detect it early.
- Apply dropout or early stopping in neural networks.
Why it matters
A model is only useful if it performs on data it has not seen. Overfitting is the most common way a model that looks excellent in development fails in deployment. It is one side of the bias-variance trade-off.
Fusion connection
A surrogate that overfits its training simulations would mislead the Hyperion design search. Kronos guards against this with held-out validation and by confirming promising candidates against full physics codes.