Active Subspaces

Active subspaces find the few input directions along which a function changes most, collapsing high-dimensional problems onto a low-dimensional ridge.

The core idea

Many functions of many variables vary strongly along only a few directions in input space and are nearly flat in the rest. The active subspace is the span of those important directions. Restricting analysis and surrogate modeling to this subspace turns an intractable high-dimensional problem into a manageable low-dimensional one.

Construction from gradients

Kronos motion — which application

Form the matrix C = E[ grad_f grad_f^T ], the average outer product of the function's gradient over the input distribution. Its eigenvectors are ordered by the eigenvalues, which measure the average squared directional derivative. A gap in the eigenvalue spectrum separates active directions (large eigenvalues) from inactive ones (near zero), defining the subspace dimension.

python

import numpy as np
# grads: (N, d) sampled gradients of f
C = grads.T @ grads / grads.shape[0]
vals, vecs = np.linalg.eigh(C)
vals, vecs = vals[::-1], vecs[:, ::-1]
# active subspace = first k columns of vecs where a spectral gap appears
k = 1
W = vecs[:, :k]
y = X @ W  # active variables

When gradients are unavailable

Gradients can come from adjoint solvers, automatic differentiation, or finite differences. When none are affordable, gradient-free variants estimate C from function values using local linear models or from a fitted surrogate's derivatives, at some loss of accuracy.

Using the subspace

Fit a cheap surrogate on the active variables y = W^T x
Visualize the response with a shadow plot of f against the first active variable
Run sensitivity analysis and optimization in reduced coordinates

Cautions

Active subspaces assume the function is well approximated by a ridge (a function of a few linear combinations of inputs). If the response is genuinely high-dimensional or has no clear eigenvalue gap, the method offers little, and forcing a low-dimensional fit will hide real behavior. The eigenvalue spectrum itself is the diagnostic: report it, and only reduce when a gap is present.