SHAP Values
SHAP attributes a prediction to its features using Shapley values, a fair-division rule from cooperative game theory.
Fair credit for a prediction
SHAP (SHapley Additive exPlanations) explains a single prediction by assigning each feature a contribution that sums, with a baseline, to the model output. It borrows the Shapley value from cooperative game theory, which distributes a team payoff among players by their fair marginal contributions, treating features as players and the prediction as the payoff.
The Shapley computation
A feature Shapley value is its average marginal contribution across all possible orderings in which features are added to the model. Averaging over orderings is what makes the attribution fair and uniquely satisfies desirable axioms. Exact computation is exponential in the number of features, so SHAP relies on efficient approximations.
Practical estimators
- KernelSHAP: a model-agnostic weighted linear regression over feature subsets
- TreeSHAP: an exact polynomial-time algorithm for tree ensembles
- DeepSHAP and GradientSHAP: approximations tailored to neural networks
Guarantees and cautions
SHAP uniquely satisfies local accuracy (contributions sum to the output), missingness, and consistency (if a feature matters more in a new model, its attribution does not decrease). Aggregating absolute SHAP values across a dataset yields a principled global importance ranking, and dependence plots reveal feature effects and interactions.
The main caution is correlated features: SHAP must assume how absent features are distributed, and different background choices (marginal versus conditional) give different, sometimes counterintuitive, attributions. Attributions describe the model behavior, not causation in the world, so a large SHAP value does not license a causal claim. SHAP is the most widely used member of the interpretability toolkit, complementing the local LIME approach.