Deep Kernel Learning (Emulator Error Control)
Deep kernel learning composes a neural feature extractor with a Gaussian-process kernel, giving calibrated uncertainty on top of learned representations -- powerful, but with real cautions for emulator error control.
Learned features, GP uncertainty
A Gaussian process is only as good as its kernel. Deep kernel learning (Wilson et al., 2016) replaces a hand-chosen kernel with k(g(x), g(x')), where g is a neural network trained end-to-end with the GP marginal likelihood. The network learns a representation in which a simple base kernel works well, while the GP layer supplies calibrated predictive variance.
Why it helps a surrogate
For emulators of expensive simulations, the payoff is flexibility plus uncertainty: the network captures nonlinear, high-dimensional structure that a stationary kernel misses, and the GP still returns an error bar. It pairs naturally with multi-fidelity schemes, where a learned feature map can align low- and high-fidelity inputs -- see nonlinear multi-fidelity GP.
The error-control cautions
- The feature map can distort distances, collapsing far-apart inputs and producing overconfident, miscalibrated variance.
- With few high-fidelity points the network overfits; the GP then trusts a bad representation.
- Marginal-likelihood training does not guarantee calibration -- it must be checked on held-out runs.
The honest rule: validate the emulator's coverage on withheld high-fidelity data, and prefer a plain GP or a simpler kernel when it already explains the response. Added flexibility is only worth it when it measurably reduces predictive error without inflating calibration error.