Ensemble Learning
Combining multiple models so their collective prediction beats any single one.
Definition
Ensemble learning combines the predictions of several models to produce a result more accurate and robust than any individual model. The gains come from errors that are partly independent and so partially cancel.
Ensembles pay off most when their members make different errors, so diversity, through different algorithms, data samples, or random seeds, matters as much as the strength of any one model. Combining highly correlated models yields little improvement over a single one.
The gains from ensembling come at the cost of computation and interpretability, since predictions now require running several models and explaining a committee is harder than explaining one member. Where those costs are acceptable, ensembles are among the most reliable ways to improve accuracy and, through the spread of member predictions, to obtain a rough uncertainty estimate that a single model cannot provide.
Main strategies
- Bagging: train models on resampled data and average (random forests).
- Boosting: train models sequentially to fix prior errors (gradient boosting).
- Stacking: train a meta-model to combine base models.
Why it matters
Ensembles dominate structured-data competitions and are a reliable way to reduce variance and improve generalization. The cost is greater computation and reduced interpretability compared with a single model.
Fusion connection
An ensemble of surrogates gives Kronos both a prediction and a spread across members, an informal uncertainty estimate that flags Hyperion configurations where the models disagree and physics review is warranted.