Quantile Regression Uncertainty
Quantile regression predicts conditional quantiles directly, capturing the shape of the output distribution without assuming it is Gaussian.
Predicting quantiles
Instead of predicting only a conditional mean, quantile regression estimates the value below which a given fraction of outcomes fall, for example the 5th, 50th, and 95th percentiles. The gap between an upper and lower quantile forms a prediction interval that need not be symmetric, unlike a mean-plus-standard-deviation interval.
The pinball loss
Quantile tau is fit by minimizing the pinball (quantile) loss, which penalizes under-prediction and over-prediction asymmetrically: L_tau(y, q) = max(tau*(y-q), (tau-1)*(y-q)). Minimizing its expectation yields the true conditional tau-quantile. Any model, from linear regression to gradient-boosted trees to neural networks, can be trained with this loss.
Advantages
- Captures heteroscedastic and skewed noise directly
- Makes no Gaussian assumption on the output
- Interpretable intervals with a clear probabilistic meaning
The crossing problem
Quantiles fit independently can cross, with a lower quantile predicted above a higher one, which is nonsensical. Remedies include jointly training all quantiles with monotonicity constraints, rearranging predicted quantiles post-hoc, or fitting a monotone model.
Combining with conformal prediction
Quantile regression gives sharp, adaptive intervals but its coverage is only as good as the model. Conformalized quantile regression calibrates the intervals on a held-out set to restore a finite-sample coverage guarantee, combining the adaptivity of quantile regression with the rigor of conformal prediction. This pairing is a strong default for surrogate prediction intervals.