BOOSTING THE CYCLE COUNTING POWER OF GRAPH NEURAL NETWORKS WITH I 2 -GNNS

Abstract

Message Passing Neural Networks (MPNNs) are a widely used class of Graph Neural Networks (GNNs). The limited representational power of MPNNs inspires the study of provably powerful GNN architectures. However, knowing one model is more powerful than another gives little insight about what functions they can or cannot express. It is still unclear whether these models are able to approximate specific functions such as counting certain graph substructures, which is essential for applications in biology, chemistry and social network analysis. Motivated by this, we propose to study the counting power of Subgraph MPNNs, a recent and popular class of powerful GNN models that extract rooted subgraphs for each node, assign the root node a unique identifier and encode the root node's representation within its rooted subgraph. Specifically, we prove that Subgraph MPNNs fail to count more-than-4-cycles at node level, implying that node representations cannot correctly encode the surrounding substructures like ring systems with more than four atoms. To overcome this limitation, we propose I 2 -GNNs to extend Subgraph MPNNs by assigning different identifiers for the root node and its neighbors in each subgraph. I 2 -GNNs' discriminative power is shown to be strictly stronger than Subgraph MPNNs and partially stronger than the 3-WL test. More importantly, I 2 -GNNs are proven capable of counting all 3, 4, 5 and 6-cycles, covering common substructures like benzene rings in organic chemistry, while still keeping linear complexity. To the best of our knowledge, it is the first linear-time GNN model that can count 6-cycles with theoretical guarantees. We validate its counting power in cycle counting tasks and demonstrate its competitive performance in molecular prediction benchmarks.

1. INTRODUCTION

Relational and structured data are usually represented by graphs. Representation learning over graphs with Graph Neural Networks (GNNs) has achieved remarkable results in drug discovery, computational chemistry, combinatorial optimization and social network analysis (Bronstein et al., 2017; Duvenaud et al., 2015; Khalil et al., 2017; Kipf & Welling, 2016; Stokes et al., 2020; You et al., 2018; Zhang & Chen, 2018 ). Among various GNNs, Message Passing Neural Network (MPNN) is one of the most commonly used GNNs (Zhou et al., 2020; Veličković et al., 2017; Scarselli et al., 2008) . However, the representational power of MPNNs is shown to be limited by the Weisfeiler-Lehman (WL) test (Xu et al., 2018; Morris et al., 2019) , a classical algorithm for graph isomorphism test. MPNNs cannot recognize even some simple substructures like cycles (Chen et al., 2020) . It leads to increasing attention on studying the representational power of different GNNs and designing more powerful GNN models. The representational power of a GNN model can be evaluated from two perspectives. One is the ability to distinguish a pair of non-isomorphic graphs, i.e., discriminative power. Chen et al. (2019) show the equivalence between distinguishing all pairs of non-isomorphic graphs and approximating all permutation invariant functions (universal approximation). Though the discriminative power provides a way to compare different models, for most GNN models without universal approximation property, it fails to tell what functions these models can or cannot express. Another perspective is to characterize the function classes expressed by a GNN model. In this regard, Chen et al. ( 2020) discusses the WL test's power of counting general graph substructures. Graph substructures are important as they are closely related to tasks in chemistry (Deshpande et al., 2002; Jin et al., 2018; Murray & Rees, 2009 ), biology (Koyutürk et al., 2004) and social network analysis (Jiang et al., 2010) . Particularly, cycles play an essential role in organic chemistry. Different types of rings impact the compounds' stability, aromaticity and other chemical properties. Therefore, studying the approximating power of counting substructures, especially cycles, provides a fine-grained and intuitive description of models' representational power and gives insight to real-world practices. Nevertheless, the difficulty of counting cycles is usually underestimated. Although You et al. (2021) claim that ID-GNNs can count arbitrary cycles at node level, the proof turns out to be incorrect, since it confuses walks with paths (a cycle is a closed path without repeated nodes while walks allow repeated nodes). In fact, even powerful 2-FWL test with cubic complexity can only count up to 7-cycles (Fürer, 2017; Arvind et al., 2020) . The difficulty makes us question whether existing powerful models, such as ID-GNNs, can count cycles properly. ID-GNNs can be categorized into a new class of GNNs named Subgraph GNNs (Cotta et al., 2021; Bevilacqua et al., 2021; Zhang & Li, 2021; You et al., 2021; Zhao et al., 2021; Papp et al., 2021) . The core idea is to decompose a graph into a bag of subgraphs and encode the graph by aggregating subgraph representations, though the strategy of extracting subgraphs varies. See Frasca et al. Theoretically, the discriminative power of existing Subgraph GNNs is known to be strictly stronger than WL test and weaker than 3-WL test (Frasca et al., 2022) . However, it is fair to say we still do not know the approximation power of Subgraph GNNs in terms of counting substructures. • To overcome the limitation, we propose I 2 -GNNs that extend Subgraph MPNNs by using multiple node identifiers. The main idea is to tie each subgraph with a node pair, including a root node and one of its neighbors. For each resulting subgraph we label the node pair with unique identifiers, which is the key to increasing the representational power. • Theoretically, we prove that I 2 -GNNs are strictly more powerful than WL test and Subgraph MPNNs, and partially more powerful than 3-WL test. Importantly, we prove I 2 -GNNs can count all cycles with length less than 7, covering important ring systems like benzene rings in chemistry. Given bounded node degree, I 2 -GNNs have linear space and time complexity w.r.t. the number of nodes, making it very scalable in real-world applications. To our best knowledge, I 2 -GNN is the first linear-time GNN model that can count 6-cycles with rigorous theoretical guarantees. Finally, we validate the counting power of I 2 -GNNs on both synthetic and realworld datasets. We demonstrate the highly competitive results of I 2 -GNNs on multiple open benchmarks compared to other state-of-the-art models.

2. PRELIMINARIES

Let G = (V, E) be a simple and undirected graph where V = {1, 2, 3, ..., N } is the node set and E ⊆ V × V is the edge set. We use x i to denote attributes of node i and e i,j to denote attributes of edge (i, j). We denote the neighbors of node i by N (i) ≜ {j ∈ V |(i, j) ∈ E}. A subgraph G S = (V S , E S ) of G is a graph with V S ⊆ V and E S ⊆ E. In this paper, we focus on counting paths and cycles. A (simple) L-path is a sequence of edges [(i 1 , i 2 ), (i 2 , i 3 ), ..., (i L , i L+1 )] such that all nodes are distinct: i 1 ̸ = i 2 ̸ = ... ̸ = i L+1 . A (simple) Lcycle is an L-path except that i 1 = i L+1 . Obviously, we have L ≥ 3 for any cycle. Two paths/cycles



(2022); Papp & Wattenhofer (2022) for detailed discussions. Subgraph GNNs have demonstrated their impressive performance by achieving state-of-the-art results on multiple open benchmarks.

Main contributions. In our work, we propose to study the representational power of Subgraph GNNs via the ability to count a specific class of substructures-cycles and paths, because they are the bases to represent some important substructures such as ring systems in chemistry. We focus on Subgraph MPNNs, a subclass of Subgraph GNNs covering Cotta et al. (2021); Zhang & Li (2021); You et al. (2021). Our main contribution include • We prove that Subgraph MPNNs can count 3-cycles and 4-cycles, but cannot count 5-cycle or any longer cycles at node level. This result is unsatisfying because only a small portion of ring systems are 4-cycles. It also negates the previous proposition that ID-GNNs can use node representations to count arbitrary cycles (You et al., 2021).

