Information Theory for Sensor Selection

Which diagnostics to trust, place, or prioritize is an information question; entropy and mutual information quantify how much each sensor tells the twin.

Measuring information

Not all diagnostics are equally informative about the states the twin needs. Information theory quantifies this: entropy measures uncertainty in a quantity, and mutual information measures how much observing a sensor reduces uncertainty about the state. These let the stack rank, place, and prioritize diagnostics on a principled basis rather than by intuition.

Entropy and mutual information:

  H(X)   = - sum p(x) log p(x)            (uncertainty in X)
  I(X;Y) = H(X) - H(X|Y)                  (info Y gives about X)
         = KL( p(x,y) || p(x)p(y) )

Gaussian case (state x, measurement y):
  I(x;y) = 0.5 * log( det(P_prior) / det(P_post) )

Optimal experiment design

Choosing a sensor set to maximize information about the state is Bayesian optimal experiment design. Common criteria come from the posterior covariance: D-optimality maximizes the information determinant, A-optimality minimizes total posterior variance. These connect directly to the observability Gramian - sensors are chosen to make safety-critical states strongly observable.

# greedy D-optimal sensor selection (schematic)
selected = []
for _ in range(budget):
    best = argmax_over_candidates(
        lambda s: logdet(info_matrix(selected + [s])))
    selected.append(best)     # add sensor with most added information
# maximizes reduction in posterior uncertainty

Use across the stack

Mutual information ranks which sensors most reduce uncertainty in shape, current, or plug potential, guiding both diagnostic placement and how the estimator weights channels. It also drives active learning: the twin proposes the measurement or scenario that would most reduce model uncertainty. On the burner, information analysis exposes how little the sparse, extrapolative diagnostic picture constrains the state - reinforcing, not hiding, the regime caveat.

• Entropy: how uncertain a quantity is.
• Mutual information: how much a sensor reduces that uncertainty.
• D-/A-optimality: principled sensor placement and prioritization.
• Active learning: pick the most informative next measurement.

Information theory ties the estimation and optimization layers together: it decides where to look, and observability decides whether what is seen is enough - both feeding the twin's honest account of what it does and does not know.