THE EFFECTIVE COALITIONS OF SHAPLEY VALUE FOR INTEGRATED GRADIENTS Anonymous

Abstract

Many methods aim to explain deep neural networks (DNN) by attributing the prediction of DNN to its input features, like Integrated Gradients and Deep Shap, which both have critical baseline problems. Previous studies pursue a perfect but intractable baseline value, which is hard to find and has a very high computational cost, limiting the application range of these baseline methods. In this paper, we propose to find a set of baseline values corresponding to Shapley values which are easier to be found and have a lower computation cost. To solve computation dilemma of Shapley value, we propose Effective Shapley value (ES), a proportional sampling method to well simulate the ratios between the Shapley values of features and then propose Shapley Integrated Gradients (SIG) to combine Integrated Gradients with ES, to achieve a good balance between efficiency and effectiveness. Experiment results show that our ES method can well and stably approximate the ratios between Shapley values, and our SIG method has a much better and more accurate performance than common baseline values with similar computational costs.

1. INTRODUCTION

Deep Learning (DL) has exhibited significant success in various tasks, such as computer vision and reinforcement learning. Unfortunately, under the curse of transparency-performance trade-off, it's difficult to understand the intrinsic working logic of DL. Attributing the prediction of a deep network to its input features is one of the most popular methods in DL evaluation domain, such as DeepLift (Shrikumar et al. (2017) ), Integrated Gradients (Sundararajan et al. (2017) ) and Deep Shap (Lundberg & Lee (2017) ). All these methods have a crucial problem how to choose a perfect baseline as a benchmark for input. As mentioned in Frye et al. ( 2020 2021) learns baseline values corresponding to a set of features. Those methods try to approximate the perfect baseline value. However, it's difficult to find a baseline value that perfectly satisfies the two principles for various inputs in practice. For example, in the field of computer vision, zero baseline value is a common baseline value, seen as bringing no additive information. But in facial expression code task for Asian people whose eyes are black is no longer suitable for zero baseline value, owing to the black area around the eyes bringing additive information. It's ideal to use a transparent image as baseline, which is impossible to achieve in computer. Therefore, though many methods try to find a perfect baseline value, most people still use empirical baseline values based on experience, which leads to unsatisfactory and unstable results. Instead of finding a perfect baseline value, we propose to find a set of informative baseline values, which can be found easier and have a much low computation. Shapley value (Shapley (1951) ) is computed as summation of marginal difference for all coalitions and Shapley value can accurately reflect the contributions of features. Aas et al. ( 2021) holds the view that Shapley value can explain the difference between prediction and global average prediction. What's more, most coalitions in Shapley value are informative and we can simply remove some features to get an informative coalition, which means can be found easier. So we propose to find a set of informative baseline values associated with Shapley value. However, the computation of Shapley value needs to iterate over all combinations, which is exponential with respect to the number of features. For the computation dilemma of Shapley value, we propose a proportional sampling method to approximate the ratios between Shapley value and propose Shapley Integrated Gradients (SIG) to combine Shapley value and Integrated Gradients, achieving a good balance between efficiency and effectiveness. Integrated Gradients has a much faster computation process compared with Shapley value but we discover that though Ren et al. ( 2021) has pointed out that Integrated Gradients is a simulation of Aumann Shapley value alongside a special calculation path, Integrated Gradients takes a shortcut compared with calculation path of Shapley value and it will lead to an unsatisfactory and unstable explanation. To verify the effectiveness of our sample methods, we conduct experiments on three typical tasks: human-defined function to verify the validity of our method to simulate ratio between Shapley value of players. facial expression code & image classification, to verify that our methods have better performance compared to zero baseline method or mean baseline methods; Our contributions can be summarized as follows: (1) We discover that Integrated Gradients takes a shortcut compared with calculation path of Shapley value; (2) We propose an effective proportional sampling method Effective Shapley value to approximate the ratios between Shapley values and design experiments to verify the effectiveness and preciseness of our methods; (3) We propose Shapley Integrated Gradients which combines Integrated Gradients with Effective Shapley value and achieve a balance between efficiency and effectiveness.

2. RELATED WORK

Most previous studies focused on finding a perfect baseline value, while our proposed method try to find a set of baseline values that are informative and easier to obtain. In our proposed methods, the selection of baseline values is based on the calculation path of Shapley value. 



), the quality of baselines determines the quality of explanations for DL. Ren et al. (2021) considers there are two key requirements: (i) baseline values should remove all information represented by origin variable values and (ii) baseline values shouldn't bring in new/abnormal information. Some studies provide empirical baseline values based on actual experience. Ancona et al. (2019) set baseline values as zero, Dabkowski & Gal (2017) set baseline values as mean value over many samples and usually people randomly select some samples from datasets. While other studies try to find a more reasonable baseline value. Fong & Vedaldi (2017) makes baseline values smoothed by blurring the input image with Gaussian noise. Frye et al. (2020) sets the baseline value of a pixel with surrounding pixels. Ren et al. (

Shapely value. Shapley value(Shapley (1951)  was proposed to distribute contributions to players, assuming that they are collaborating. It's the only distribution with desirable properties, linearity, nullity, symmetry, and efficiency axioms.Aumann & Shapley (2015)  extended the concept of Shapley value to infinite game. Some previous studies used Shapley value for model explanation.Lundberg & Lee (2017)  proposed Shapley Additive exPlanations(SHAP), a model explanation method with Shapley value. The SHAP regards the feature as a player in game, regards model as utility function, and uses chain rule to reduce computational complexity. Based on SHAP, Lundberg et al. (2018) continued to propose TreeSHAP, a method for tree model reducing complexity from O(TL2 M) to O(TLD 2). It's worth noting that there is also a baseline problem in SHAP. To simplify computation, Ghorbani & Zou (2019) used Monte Carlo Sampling and gradient-based methods to efficiently estimate data Shapley values. Ancona et al. (2019) sampled from distribution of coalition size of k and then average all these marginal contributions as an approximation of Shapley value. Ghorbani & Zou (2020) proposed a new multi-armed bandit algorithm to explain Neuron's Shapley value. Integrated Gradients. Integrated Gradients was proposed by Sundararajan et al. (2017) to combine implementation invariance of gradients along with the sensitivity of techniques, which also needs a crucial baseline value. Merrick & Taly (2020); Kumar et al. (2020); Binder et al. (2016); Shrikumar et al. (2017) provided their experiential guidance of selecting baseline values, without providing any theoretical guidance. And Chen et al. (2021) regarded baseline values as background distribution, which is similar to our view.3 PRELIMINARIESShapley value. Let us consider a game with n players and F = S|S ⊆ 2 n means all subsets of players. Game will return a reward corresponding to the coalition S through utility function v.

