Shapley Values
Summary
The reference chapter can be found in this book.
- Shapley Values (SV) explain prediction via considering each feature as a player and prediction as the payout and deals with how to fairly distribute the payout among the players
- has solid base in coalitional game theory
- gain (payout) here is the difference between actual prediction and average
- SV for an instance is the average marginal contribution of a feature value across all possible coalitions (weighted average, to be precise)
- marginal contribution = difference between prediction with and without this feature value
- computing SV for linear models is very straightforward
- SV satisfy the following properties
- Efficiency - feature contributions add upto difference between predicted and mean prediction
- Symmetry - SV of 2 feature values should be the same if they contribute equally for all coalitions
- Dummy - if a feature value does not contribute to the output, then its value should be 0
- Additivity - for ensemble models, final SV can be computed by adding up the individual SV from each of the underlying models
Computating SV
$$\phi_j = \sum_{S \subset {x} - {x_j}} \frac{len(S)! (p - len(S) - 1)!}{p!} (val(S \bigcup {x_j}) - val(S))$$
$$val(S) = \int_{x \notin S} f(x_1, ..., x_p) - E[f(X)]$$
Where:
- $$\phi_j$$ - SV for j-th feature value
- $$p$$ - number of features
- $$f$$ - the model
Approximating SV via Monte-Carlo sampling
$$\phi'j = \frac{1}{M} \sum{m=1}^M f(x_{+j}^m) - f(x_{-j}^m)$$
Where:
- $$M$$ - number of samples
- $$f(x_{+j}^m)$$ - prediction where the feature values are first randomly permuted and then the first j feature values are kept from the input sample while the rest are randomly picked from another sample in the dataset.
- $$f(x_{-j}^m)$$ - same as above but with j-th feature value also chosen at random from another sample
Disadvantages
- suffers from inclusion of unrealistic feature values
- computationally super expensive
- explanation involves all the features, which can be difficult for humans to further interpret, as opposed to explainer models like LIME.