RETHINKING THE EXPRESSIVE POWER OF GNNS VIA GRAPH BICONNECTIVITY

Abstract

Designing expressive Graph Neural Networks (GNNs) is a central topic in learning graph-structured data. While numerous approaches have been proposed to improve GNNs in terms of the Weisfeiler-Lehman (WL) test, generally there is still a lack of deep understanding of what additional power they can systematically and provably gain. In this paper, we take a fundamentally different perspective to study the expressive power of GNNs beyond the WL test. Specifically, we introduce a novel class of expressivity metrics via graph biconnectivity and highlight their importance in both theory and practice. As biconnectivity can be easily calculated using simple algorithms that have linear computational costs, it is natural to expect that popular GNNs can learn it easily as well. However, after a thorough review of prior GNN architectures, we surprisingly find that most of them are not expressive for any of these metrics. The only exception is the ESAN framework (Bevilacqua et al., 2022), for which we give a theoretical justification of its power. We proceed to introduce a principled and more efficient approach, called the Generalized Distance Weisfeiler-Lehman (GD-WL), which is provably expressive for all biconnectivity metrics. Practically, we show GD-WL can be implemented by a Transformer-like architecture that preserves expressiveness and enjoys full parallelizability. A set of experiments on both synthetic and real datasets demonstrates that our approach can consistently outperform prior GNN architectures.

1. INTRODUCTION

Graph neural networks (GNNs) have recently become the dominant approach for graph representation learning. Among numerous architectures, message-passing neural networks (MPNNs) are arguably the most popular design paradigm and have achieved great success in various fields (Gilmer et al., 2017; Hamilton et al., 2017; Kipf & Welling, 2017; Veličković et al., 2018 ). However, one major drawback of MPNNs lies in the limited expressiveness: as pointed out by Xu et al. (2019) ; Morris et al. (2019) , they can never be more powerful than the classic 1-dimensional Weisfeiler-Lehman (1-WL) test in distinguishing non-isomorphic graphs (Weisfeiler & Leman, 1968 ). This inspired a variety of works to design provably more powerful GNNs that go beyond the 1-WL test. One line of subsequent works aimed to propose GNNs that match the higher-order WL variants (Morris et al., 2019; 2020; Maron et al., 2019c; a; Geerts & Reutter, 2022) . While being highly expressive, such an approach suffers from severe computation/memory costs. Moreover, there have been concerns about whether the achieved expressiveness is necessary for real-world tasks (Veličković, 2022) . In light of this, other recent works sought to develop new GNN architectures with improved expressiveness while still keeping the message-passing framework for efficiency (Bouritsas et al., 2022; Bodnar et al., 2021b; a; Bevilacqua et al., 2022; Wijesinghe & Wang, 2022 , and see Appendix A for more recent advances). However, most of these works mainly justify their expressiveness by giving toy examples where WL algorithms fail to distinguish, e.g., by focusing on regular graphs. On the theoretical side, it is quite unclear what additional power they can systematically and provably gain. More fundamentally, to the best of our knowledge (see Appendix D.1), there is still a lack of principled and convincing metrics beyond the WL hierarchy to formally measure the expressive power and to guide the design of provably better GNN architectures. In this paper, we systematically study the problem of designing expressive GNNs from a novel perspective of graph biconnectivity. Biconnectivity has long been a central topic in graph theory (Bollobás, 1998) . It comprises a series of important concepts such as cut vertex (articulation point), cut edge (bridge), biconnected component, and block cut tree (see Section 2 for formal definitions). Intuitively, biconnectivity provides a structural description of a graph by decomposing it into disjoint sub-components and linking them via cut vertices/edges to form a tree structure (cf. Figure 1(b, c )). As can be seen, biconnectivity purely captures the intrinsic structure of a graph. The significance of graph biconnectivity can be reflected in various aspects. Firstly, from a theoretical point of view, it is a basic graph property and is linked to many fundamental topics in graph theory, ranging from path-related problems to network flow (Granot & Veinott Jr, 1985) and spanning trees (Kapoor & Ramesh, 1995), and is highly relevant to planar graph isomorphism (Hopcroft & Tarjan, 1972) . Secondly, from a practical point of view, cut vertices/edges have substantial values in many real applications. For example, chemical reactions are highly related to edge-biconnectivity of the molecule graph, where the breakage of molecular bonds usually occurs at the cut edges and each biconnected component often remains unchanged after the reaction. As another example, social networks are related to vertex-biconnectivity, where cut vertices play an important role in linking between different groups of people (biconnected components). Finally, from a computational point of view, the problems related to biconnectivity (e.g., finding cut vertices/edges or constructing block cut trees) can all be efficiently solved using classic algorithms (Tarjan, 1972) , with a computation complexity equal to graph size (which is the same as an MPNN). Therefore, one may naturally expect that popular GNNs should be able to learn all things related to biconnectivity without difficulty. Unfortunately, we show this is not the case. After a thorough analysis of four classes of representative GNN architectures in literature (see Section 3.1), we find that surprisingly, none of them could even solve the easiest biconnectivity problem: to distinguish whether a graph has cut vertices/edges or not (corresponding to a graph-level binary classification). As a result, they obviously failed in the following harder tasks: (i) identifying all cut vertices (a node-level task); (ii) identifying all cut edges (an edge-level task); (iii) the graph-level task for general biconnectivity problems, e.g., distinguishing a pair of graphs that have non-isomorphic block cut trees. This raises the following question: can we design GNNs with provable expressiveness for biconnectivity problems? We first give an affirmative answer to the above question. By conducting a deep analysis of the recently proposed Equivariant Subgraph Aggregation Network (ESAN) (Bevilacqua et al., 2022) , we prove that the DSS-WL algorithm with node marking policy can precisely identify both cut vertices and cut edges. This provides a new understanding as well as a strong theoretical justification for the expressive power of DSS-WL and its recent extensions (Frasca et al., 2022) . Furthermore, we give a fine-grained analysis of several key factors in the framework, such as the graph generation policy and the aggregation scheme, by showing that neither (i) the ego-network policy without marking nor (ii) a variant of the weaker DS-WL algorithm can identify cut vertices. However, GNNs designed based on DSS-WL are usually sophisticated and suffer from high computation/memory costs. The main contribution in this paper is then to give a principled and efficient way to design GNNs that are expressive for biconnectivity problems. Targeting this question, we restart from the classic 1-WL algorithm and figure out a major weakness in distinguishing biconnectivity: the lack of distance information between nodes. Indeed, the importance of distance information is theoretically justified in our proof for analyzing the expressive power of DSS-WL. To this end, we introduce a novel color refinement framework, formalized as Generalized Distance Weisfeiler-Lehman (GD-WL), by directly encoding a general distance metric into the WL aggrega-



Figure 1: An illustration of edge-biconnectivity and vertex-biconnectivity. Cut vertices/edges are outlined in bold red. Gray nodes in (b)/(c) are edge/vertex-biconnected components, respectively.

