Active Learning for Surrogates
Active learning chooses where to run an expensive simulation next, so a surrogate improves fastest with the fewest costly evaluations.
Spending evaluations wisely
When each simulation is expensive, the placement of the next evaluation matters enormously. Active learning is the strategy of selecting new sample points to maximize what the surrogate learns per run, rather than sampling blindly. Guided by the surrogate's own uncertainty, it concentrates effort where it will most improve the model or most advance the goal.
Acquisition functions
The rule that scores candidate points is the acquisition function. For pure model improvement, it favors points where the surrogate is most uncertain. For optimization, it balances exploring uncertain regions against exploiting regions that look promising, as in expected improvement and upper confidence bound criteria used in Bayesian optimization. The next evaluation goes where the acquisition function is highest.
Exploration versus exploitation
- Pure exploration: sample the most uncertain region to refine the whole surrogate
- Pure exploitation: sample near the current best to sharpen the optimum
- Bayesian optimization balances the two automatically
- Batch methods pick several diverse points for parallel simulation runs
The loop
Active learning proceeds in a cycle: fit the surrogate to current data, use the acquisition function to choose the next point, run the expensive simulation there, add the result, and repeat. Because it needs the surrogate's uncertainty, methods with calibrated uncertainty, such as Gaussian processes and ensembles, are the natural backbone of the loop.
Why it matters
Active learning can reach a target accuracy or find an optimum with a small fraction of the evaluations that uniform sampling would need. For design studies built on costly physics simulations, this efficiency is often the difference between a study that is feasible and one that is not, which is why it is central to surrogate-based engineering.