Distributionally Robust Optimization
Distributionally robust optimization hedges against uncertainty in the probability distribution itself, optimizing the worst case over a set of plausible distributions.
Between stochastic and robust
Stochastic programming assumes the exact probability distribution is known; robust optimization ignores probability and protects against every value in a set. Distributionally robust optimization (DRO) sits between them: it assumes the true distribution is unknown but lies within an ambiguity set of plausible distributions, and optimizes the worst-case expected cost over that set. This addresses the real difficulty that distributions are estimated from limited data and may be wrong.
The ambiguity set
The ambiguity set collects all distributions consistent with what is known. Two families dominate. Moment-based sets contain every distribution matching an estimated mean and covariance (or bounds on them). Metric-based sets contain every distribution within a chosen statistical distance of the empirical distribution, most commonly a Wasserstein ball of radius epsilon around the observed samples. The radius shrinks as more data arrives.
The formulation
DRO solves minimize over x of ( max over P in the ambiguity set of E_P[ F(x, xi) ] ). The inner maximization asks: among all plausible distributions, which makes decision x look worst? The outer minimization picks the decision whose worst plausible case is best. A key result is that Wasserstein DRO often reduces to a tractable convex problem plus a regularization term, linking robustness directly to regularization in machine learning.
- Moment ambiguity: match estimated mean and covariance, gives SDP/SOCP
- Wasserstein ambiguity: ball of transport distance, gives convex regularized programs
- Phi-divergence ambiguity: KL or chi-squared balls around the empirical law
- Radius as a knob: tune protection against distribution shift
Why it matters
DRO produces solutions that degrade gracefully when the deployment distribution differs from the training distribution, a common failure mode of methods that trust an empirical distribution exactly. It offers finite-sample performance guarantees and, through the regularization connection, explains why certain regularized estimators are robust. It is used where data is scarce or the future distribution may drift from the past.