Overfitting and Regularization
Why models memorize small fusion datasets, and the techniques that make them generalize.
What overfitting is
A model overfits when it fits the training data, including its noise and quirks, so closely that it fails on new data. With the small datasets common in fusion, overfitting is a constant threat: a flexible model can memorize a few hundred shots and learn nothing that generalizes.
Signs of it
- Training error much lower than validation error
- Performance that collapses on a new campaign or machine
- Sensitivity to tiny changes in the training set
Regularization techniques
Regularization constrains model complexity so it captures signal, not noise. Weight penalties (L1, L2) shrink parameters; dropout randomly disables units during training; early stopping halts before memorization; and data augmentation effectively enlarges the dataset. Simpler model families and fewer parameters help directly.
The bias-variance trade-off
Too simple a model underfits, missing real structure; too complex a model overfits, chasing noise. Regularization tunes this balance. In fusion, with scarce data, the balance usually favors simpler, more constrained models than a data-rich domain would use.
Validation done right
Detecting and controlling overfitting requires honest validation: split by shot or campaign, not by random time slice; never tune on the test set; and prefer cross-validation to a single split when data are scarce. A model that looks excellent under leaky validation and fails in practice is the recurring cautionary tale. The discipline of proper validation is what makes small-data fusion ML trustworthy.