DAG MATTERS! GFLOWNETS ENHANCED EX-PLAINER FOR GRAPH NEURAL NETWORKS

Abstract

Uncovering rationales behind predictions of graph neural networks (GNNs) has received increasing attention over the years. Existing literature mainly focus on selecting a subgraph, through combinatorial optimization, to provide faithful explanations. However, the exponential size of candidate subgraphs limits the applicability of state-of-the-art methods to large-scale GNNs. We enhance on this through a different approach: by proposing a generative structure -GFlowNetsbased GNN Explainer (GFlowExplainer), we turn the optimization problem into a step-by-step generative problem. Our GFlowExplainer aims to learn a policy that generates a distribution of subgraphs for which the probability of a subgraph is proportional to its' reward. The proposed approach eliminates the influence of node sequence and thus does not need any pre-training strategies. We also propose a new cut vertex matrix to efficiently explore parent states for GFlowNets structure, thus making our approach applicable in a large-scale setting. We conduct extensive experiments on both synthetic and real datasets, and both qualitative and quantitative results show the superiority of our GFlowExplainer.

1. INTRODUCTION

Graph Neural Networks (GNNs) have received widespread attention due to the springing up of graph-structured data in real-world applications, such as social networks and chemical molecules Zhang et al. (2020) . Various graph related task are widely studied including node classification Henaff et al. (2015) ; Liu et al. (2020) and graph classification Zhang et al. (2018) . However, uncovering rationales behind predictions of graph neural networks (GNNs) is relatively less explored. Recently, some explanation approaches for GNNs have gradually stepped into the public eye. There are two major branches of them: instance-level explanations and model-level explanations Yuan et al. (2022) . In this paper, we mainly focus on instance-level explanations. Instance-level approaches explain models by identifying the most critical input features for their predictions. They have four sub-branches: Gradients/Features-based Zhou et al. ( 2016 (2021) apply reinforcement learning (RL) to model-level and instance-level explanations. However, the pioneering works have some drawbacks. Perturbation-based approaches return the discrete edges for explanations, which are not as intuitive as graph generation-based approach, which could provide connected graphs. However, the task of searching connected subgraphs is a combinatorial problem, and the potential candidates increase exponentially, making most current approaches inefficient and intractable in large-scale settings. In addition, current research consider Monte-Carlo tree search, which has high variance and ignores the fact that graph is an unordered set. This could lead to a loss of sampling efficiency and effectiveness, i.e., the approaches fail to consolidate information of sampled trajectories that form the same subgraph with different sequences. To address the above issues, we take advantage of the strong generation property of Generative Flow Networks (GFlowNets) Bengio et al. (2021b) and cast the combinatorial optimization problem as a generation problem. Unlike the previous work, which focus on the maximization of mutual information, our insight is to learn a generative policy that generates a distribution of connected subgraphs with probabilities proportional to their mutual information. We called this approach GFlowExplainer, which could overcome the current predicament for the following reasons. First, it has a stronger exploration ability due to its flow matching condition, helping us to avoid the trap of suboptimal solutions. Second, in contrast to previous tree search or node sequence modeling, GFlowExplainer consolidate information from sampled trajectories generating the same subgraph with different sequences. This critical difference could largely increase the utilization of generated samples, and hence improve the performance. Moreover, by introducing a cut vertex matrix, GFlow-Explainer could be applied in large-scale settings and achieve better performance with fewer training epochs. We summarize the main contributions as follows. Main Contributions: 1) We propose a new hand-crafted method for GNN explanation via GFlowNet frameworks to sample from a target distribution with the energy proportional to the predefined score function; 2) We take advantage of the DAG structure in GFlowNets to connect the trajectories of outputting the same graph but different node sequences. Therefore, without any pre-training strategies, we can significantly improve the effectiveness of our GNN explanations; 3) Considering relatively cumbersome valid parent state explorations in GFlowNets because of the connectivity constraint of the graph, we introduce the concept of cut vertex and propose a more efficient cut vertex criteria for dynamic graphs, thus speeding up the whole process; 4) We conduct extensive experiments to show that GFlowExplainer can outperform current state-of-the-art approaches.

2. RELATED WORK

Graph Neural Networks: Graph neural networks (GNNs) are developing rapidly in recent years and have been adopted to leverage the structure and properties of graphs Scarselli et al. (2008) ; Sanchez-Lengeling et al. (2021) . Most GNN variants can be summarized with the message passing scheme, which is composed of pattern extraction and interaction modeling within each layer Gilmer et al. (2017) . These approaches aggregate the information from neighbors with different functions, such as mean/max/LSTM-pooling in GCN Welling & Kipf (2016 ), GrpahSAGE Hamilton et al. (2017 ), sum-pooling in GIN Xu et al. (2018) , attention mechanisms in GAT Velickovic et al. (2017) . SGC Wu et al. (2019) observes that the superior performance of GNNs is mainly due to the neighbor aggregation rather than feature transformation and nonlinearity, and proposed a simple and fast GNN model. APPNP Klicpera et al. (2018) shares the similar idea by decoupling feature transformation and neighbor aggregation. Generative Flow Networks: Generative flow networks Bengio et al. (2021a; b) aim to train generative policies that could sample compositional objects x ∈ D by discrete action sequences with probability proportional to a given reward function. This network could sample trajectories according to a distribution proportional to the rewards, and this feature becomes particularly important when exploration is important. The approach also differs from RL, which aims to maximize the expected return and only generates a single sequence of actions with the highest reward. Instance-level GNN Explanation: Instance-level approaches explain models by identifying the most critical input features for their predictions. Gradients/Features-based approaches, e.g., Zhou et al. (2016) ; Baldassarre & Azizpour (2019); Pope et al. (2019) , compute the gradients or map the features to the input to explain the important terms while the scores sometimes could not reflect the contributions intuitively. As for the perturbation-based approaches, GNNExplainer Ying et al. ( 2019) is the first specific design for explanation of GNNs, which formulates an optimization task to maximize the mutual information between the GNN predictions and the distribution of poten-



); Baldassarre & Azizpour (2019); Pope et al. (2019), Perturbation-based Ying et al. (2019); Luo et al. (2020); Schlichtkrull et al. (2020); Wang et al. (2020), Decompose-based Baldassarre & Azizpour (2019); Schnake et al. (2020); Feng et al. (2021) and Surrogate-based Vu & Thai (2020); Huang et al. (2022); Yuan et al. (2022). Some works such as XGNN Yuan et al. (2020) and RGExplainer Shan et al.

GFlowNets has been applied in molecule generation Bengio et al. (2021a); Jain et al. (2022), discrete probabilistic modeling Zhang et al. (2022), bayesian structure learning Deleu et al. (2022), causal discovery Li et al. (2022) and continuous control tasks Li et al. (2023).

funding

and Huawei Noah's Ark Lab. This work was completed while Wenqian Li was a member of the Huawei Noah's Ark Lab for advanced study.

