GOING BEYOND 1-WL EXPRESSIVE POWER WITH 1-LAYER GRAPH NEURAL NETWORKS Anonymous

Abstract

Graph neural networks have become the de facto standard for representational learning in graphs, and have achieved SOTA in many graph-related tasks such as node classification, graph classification and link prediction. However, it has been shown that the expressive power is equivalent maximally to Weisfeiler-Lehman Test. Recently, there is a line of work aiming to enhance the expressive power of graph neural networks. In this work, we propose a more generalized variant of neural Weisfeiler-Lehman test to enhance structural representation for each node in a graph to uplift the expressive power of any graph neural network. It is shown theoretically our method is strictly more powerful than 1&2-WL test. The Numerical experiments also show that our proposed method outperforms the standard GNNs on almost all the benchmark datasets by a large margin in most cases with significantly lower running time and memory consumption compared with other more powerful GNNs.

1. INTRODUCTION

Graph-structured data is ubiquitous in many real-world applications ranging from social network analysis Fan et al. (2019) , drug discovery Jiang et al. (2020) , personalized recommendation He et al. (2020) and bioinformatics Gasteiger et al. (2021) . In recent years, Graph Neural Networks (GNNs) have seized increasing attention due to their powerful expressiveness and have become dominant approaches for graph-related tasks. Message Passing Graph Neural Networks (MPGNNs) are the most common types of GNNs due to their efficiency and expressivity. MPGNNs can be viewed as a neural version of the 1-Weisfeiler-Lehman (1-WL) algorithm Weisfeiler & Leman (1968) , where colors are replaced by continuous feature vectors and neural networks are used to aggregate over node neighborhoods Morris et al. (2019) . By iteratively aggregating neighboring node features to the center node, MPGNNs learn node representations that encode their local structures and feature information. A graph readout function can be further leveraged to pool a whole-graph representation for downstream tasks such as graph classification. Despite the success of MPGNNs, it is proved in some recent literatures that the expressive power of MPGNNs is bounded by 1-WL isomorphism test (Morris et al., 2019; Xu et al., 2018a) , i.e, standard MPGNNs or 1-WL GNNs cannot distinguish any (sub-)graph structure that 1-WL cannot distinguish such as for any two n-node r-regular graphs, standard MPGNNs will output the same node representation. Since then, a few works have been proposed to enhance the expressivity of MPGNNs. Methods proposed by (Morris et al., 2019; Chen et al., 2019; Maron et al., 2019) aim at approximating high-dimensional WL tests. However, these methods require learning all node tuples, which are computationally expensive and not able to scale well to large-scale graphs. Another line of works augment node features to enhance the expressive power of GNNs. E.g., works proposed by (Loukas, 2019; Sato et al., 2021) inject one-hot features or random features to each node of a graph, while other works incorporate structural features to enhance expressivity of GNNs such as distance-based features (Zhang & Chen, 2019; Li et al., 2020) and counting features of certain substructures Bouritsas et al. (2022) . More recently, (Zhang & Li, 2021; Zhao et al., 2021) propose to leverage subgraph information that cannot be captured by 1-WL test to infer node representations. Concretely, instead of hashing the direct neighborhood information in 1-WL test, these methods hash the subgraph information, and therefore inject additional structural information in the learning process. These

