RETHINKING THE EXPRESSIVE POWER OF GNNS VIA GRAPH BICONNECTIVITY

Abstract

Designing expressive Graph Neural Networks (GNNs) is a central topic in learning graph-structured data. While numerous approaches have been proposed to improve GNNs in terms of the Weisfeiler-Lehman (WL) test, generally there is still a lack of deep understanding of what additional power they can systematically and provably gain. In this paper, we take a fundamentally different perspective to study the expressive power of GNNs beyond the WL test. Specifically, we introduce a novel class of expressivity metrics via graph biconnectivity and highlight their importance in both theory and practice. As biconnectivity can be easily calculated using simple algorithms that have linear computational costs, it is natural to expect that popular GNNs can learn it easily as well. However, after a thorough review of prior GNN architectures, we surprisingly find that most of them are not expressive for any of these metrics. The only exception is the ESAN framework (Bevilacqua et al., 2022), for which we give a theoretical justification of its power. We proceed to introduce a principled and more efficient approach, called the Generalized Distance Weisfeiler-Lehman (GD-WL), which is provably expressive for all biconnectivity metrics. Practically, we show GD-WL can be implemented by a Transformer-like architecture that preserves expressiveness and enjoys full parallelizability. A set of experiments on both synthetic and real datasets demonstrates that our approach can consistently outperform prior GNN architectures.

1. INTRODUCTION

Graph neural networks (GNNs) have recently become the dominant approach for graph representation learning. Among numerous architectures, message-passing neural networks (MPNNs) are arguably the most popular design paradigm and have achieved great success in various fields (Gilmer et al., 2017; Hamilton et al., 2017; Kipf & Welling, 2017; Veličković et al., 2018 ). However, one major drawback of MPNNs lies in the limited expressiveness: as pointed out by Xu et al. (2019) ; Morris et al. (2019) , they can never be more powerful than the classic 1-dimensional Weisfeiler-Lehman (1-WL) test in distinguishing non-isomorphic graphs (Weisfeiler & Leman, 1968 ). This inspired a variety of works to design provably more powerful GNNs that go beyond the 1-WL test. One line of subsequent works aimed to propose GNNs that match the higher-order WL variants (Morris et al., 2019; 2020; Maron et al., 2019c; a; Geerts & Reutter, 2022) . While being highly expressive, such an approach suffers from severe computation/memory costs. Moreover, there have been concerns about whether the achieved expressiveness is necessary for real-world tasks (Veličković, 2022) . In light of this, other recent works sought to develop new GNN architectures with improved expressiveness while still keeping the message-passing framework for efficiency (Bouritsas et al., 2022; Bodnar et al., 2021b; a; Bevilacqua et al., 2022; Wijesinghe & Wang, 2022 , and see Appendix A for more recent advances). However, most of these works mainly justify their expressiveness by giving toy examples where WL algorithms fail to distinguish, e.g., by focusing on regular graphs. On the theoretical side, it is quite unclear what additional power they can systematically and provably gain. More fundamentally, to the best of our knowledge (see Appendix D.1), there is still a lack of principled and convincing metrics beyond the WL hierarchy to formally measure the expressive power and to guide the design of provably better GNN architectures.

