RECURSIVE NEIGHBORHOOD POOLING FOR GRAPH REPRESENTATION LEARNING

Abstract

While massage passing based Graph Neural Networks (GNNs) have become increasingly popular architectures for learning with graphs, recent works have revealed important shortcomings in their expressive power. In response, several higher-order GNNs have been proposed, which substantially increase the expressive power, but at a large computational cost. Motivated by this gap, we introduce and analyze a new recursive pooling technique of local neighborhoods that allows different tradeoffs of computational cost and expressive power. First, we show that this model can count subgraphs of size k, and thereby overcomes a known limitation of low-order GNNs. Second, we prove that, in several cases, RNP-GNNs can greatly reduce computational complexity compared to the existing higher-order k-GNN and Local Relational Pooling (LRP) networks.

1. INTRODUCTION

Graph Neural Networks (GNNs) are powerful tools for graph representation learning (Scarselli et al., 2008; Kipf & Welling, 2017; Hamilton et al., 2017) , and have been successfully used in applications such as encoding molecules, simulating physics, social network analysis, knowledge graphs, and many others (Duvenaud et al., 2015; Defferrard et al., 2016; Battaglia et al., 2016; Jin et al., 2018) . An important class of GNNs is the set of Message Passing Graph Neural Networks (MPNNs) (Gilmer et al., 2017; Kipf & Welling, 2017; Hamilton et al., 2017; Xu et al., 2019; Scarselli et al., 2008) , which follow an iterative message passing scheme to compute a graph representation. Despite the empirical success of MPNNs, their expressive power has been shown to be limited. For example, their discriminative power, at best, corresponds to the one-dimensional Weisfeiler-Leman (1-WL) graph isomorphism test (Xu et al., 2019; Morris et al., 2019) , so they cannot, e.g., distinguish regular graphs. Moreover, they also cannot count any induced subgraph with at least three vertices (Chen et al., 2020) , or learn graph parameters such as clique information, diameter, or shortest cycle (Garg et al., 2020) . Still, in several applications, e.g. in computational chemistry, materials design or pharmacy (Elton et al., 2019; Sun et al., 2020; Jin et al., 2018) , we aim to learn functions that depend on the presence or count of specific substructures. To strengthen the expressive power of GNNs, higher-order representations such as k-GNNs (Morris et al., 2019) and k-Invariant Graph Networks (k-IGNs) (Maron et al., 2019) have been proposed. k-GNNs are inspired by the k-dimensional WL (k-WL) graph isomorphism test, a message passing algorithm on k-tuples of vertices, and k-IGNs are based on equivariant linear layers of a feed-forward neural network applied to the input graph as a matrix, and are at least as powerful as k-GNN. These models are provably more powerful than MPNNs and can, e.g., count any induced substructure with at most k vertices. But, this power comes at the computational cost of at least Ω(n k ) operations for n vertices. The necessary tradeoffs between expressive power and computational complexity are still an open question. The expressive power of a GNN is often measured in terms of a hierarchy of graph isomorphism tests, i.e., by comparing it to a k-WL test. Yet, there is limited knowledge about how the expressive power of higher-order graph isomorphism tests relates to various functions of interest (Arvind et al., 2020) . A different approach is to take the perspective of specific functions that are of practical interest, and quantify a GNN's expressive power via those. Here, we focus on counting induced substructures to measure the power of a GNN, as proposed in (Chen et al., 2020) . In particular, we

