GNNINTERPRETER: A PROBABILISTIC GENERATIVE MODEL-LEVEL EXPLANATION FOR GRAPH NEURAL NETWORKS

Abstract

Recently, Graph Neural Networks (GNNs) have significantly advanced the performance of machine learning tasks on graphs. However, this technological breakthrough makes people wonder: how does a GNN make such decisions, and can we trust its prediction with high confidence? When it comes to some critical fields, such as biomedicine, where making wrong decisions can have severe consequences, it is crucial to interpret the inner working mechanisms of GNNs before applying them. In this paper, we propose a model-agnostic model-level explanation method for different GNNs that follow the message passing scheme, GNNInterpreter, to explain the high-level decision-making process of the GNN model. More specifically, GNNInterpreter learns a probabilistic generative graph distribution that produces the most discriminative graph pattern the GNN tries to detect when making a certain prediction by optimizing a novel objective function specifically designed for the model-level explanation for GNNs. Compared to existing works, GNNInterpreter is more flexible and computationally efficient in generating explanation graphs with different types of node and edge features, without introducing another blackbox or requiring manually specified domain-specific rules. In addition, the experimental studies conducted on four different datasets demonstrate that the explanation graphs generated by GNNInterpreter match the desired graph pattern if the model is ideal; otherwise, potential model pitfalls can be revealed by the explanation.

1. INTRODUCTION

Graphs are widely used to model data in many applications such as chemistry, transportation, etc. Since a graph is a unique non-Euclidean data structure, modeling graph data remained a challenging task until Graph Neural Networks (GNNs) emerged (Hamilton et al., 2017; Cao et al., 2016) . As a powerful tool for representation learning on graph data, GNN achieved state-of-the-art performance on various different machine learning tasks on graphs. As the popularity of GNNs rapidly increases, people begin to wonder why one should trust this model and how the model makes decisions. However, the complexity of GNNs prevents humans from interpreting the underlying mechanism in the model. The lack of self-explainability becomes a serious obstacle for applying GNNs to real-world problems, especially when making wrong decisions may incur an unaffordable cost. Explaining deep learning models on text or image data (Simonyan et al., 2014; Selvaraju et al., 2019) has been well-studied. However, explaining deep learning models on graphs is still less explored. Compared with explaining models on text or image data, explaining deep graph models is a more challenging task for several reasons (Yuan et al., 2020b) : (i) the adjacency matrix representing the topological information has only discrete values, which cannot be directly optimized via gradientbased methods (Duval & Malliaros, 2021), (ii) in some application domains, a graph is valid only if it satisfies a set of domain-specific graph rules, so that generating a valid explanation graph to depicts the underlying decision-making process of GNNs is a nontrivial task, and (iii) graph data structure is heterogeneous in nature with different types of node features and edge features, which makes developing a one-size-fits-all explanation method for GNNs to be even more challenging. In this paper, we attempt to interpret the high-level decision-making process of GNNs and to identify potential model pitfalls, by resolving these three challenges respectively.

