MOTIF-INDUCED GRAPH NORMALIZATION

Abstract

Graph Neural Networks (GNNs) have emerged as a powerful category of learning architecture for handling graph-structured data. However, the existing GNNs usually follow the neighborhood aggregation scheme, ignoring the structural characteristics in the node-induced subgraphs, which limits their expressiveness for the downstream tasks of both the graph-and node-level predictions. In this paper, we strive to strengthen the general discriminative capabilities of GNNs by devising a dedicated plug-and-play normalization scheme, termed as Motif-induced Normalization (MotifNorm), that explicitly considers the intra-connection information within each node-induced subgraph. To this end, we embed the motif-induced structural weights at the beginning and the end of the standard BatchNorm, as well as incorporate the graph instance-specific statistics for improved distinguishable capabilities. In the meantime, we provide the theoretical analysis to support that the MotifNorm scheme can help alleviate the over-smoothing issue, which is conducive to designing deeper GNNs. Experimental results on eight popular benchmarks across all the tasks of the graph, node, as well as link property predictions, demonstrate the effectiveness of the proposed method. Our code is made available in the supplementary material.

1. INTRODUCTION

In recent years, Graph Neural Networks (GNNs) have emerged as the mainstream deep learning architectures to analyze irregular samples where information is present in the form of graphs, which usually employs the message-passing aggregation mechanism to encode node features from local neighborhood representations (Kipf & Welling, 2017; Veličković et al., 2018; Xu et al., 2019; Yang et al., 2020b; Hao et al., 2021; Dwivedi et al., 2022b) . As a powerful class of graph-relevant networks, these architectures have shown encouraging performance in various domains such as cell clustering (Li et al., 2022; Alghamdi et al., 2021 ), chemical prediction (Tavakoli et al., 2022; Zhong et al., 2022 ), social networks (Bouritsas et al., 2022; Dwivedi et al., 2022b ), traffic networks (Bui et al., 2021; Li & Zhu, 2021 ), combinatorial optimization (Schuetz et al., 2022; Cappart et al., 2021) , and power grids (Boyaci et al., 2021; Chen et al., 2022a) . However, the commonly used message-passing mechanism, i.e., aggregating the representations from neighborhoods, limits the expressive capability of GNNs to address the subtree-isomorphic phenomenon prevalent in the real world (Wijesinghe & Wang, 2022) . As shown in Figure 1(a) , subgraphs S v1 , S v2 induced by v 1 , v 2 are subtree-isomorphic, which decreases the GNNs' expressivity in graph-level and node-level prediction, i.e., (1) Graph-level: Straightforward neighborhood aggregations, ignoring the characterises of the node-induced subgraphs, lead to complete indistinguishability in the subtree-isomorphic case, which thus limits the GNNs' expressivity to be bottlenecked by the Weisfeiler-Leman (WL) (Weisfeiler & Leman, 1968) test. (2) Node-level: Under the background of over-smoothing (as illustrated in Figure 1(b) ), the smoothing problem among the root representations of subtree-isomorphic substructures will become worser when aggregating the similar representations from their neighborhoods without structural characterises be considered. In this paper, we strive to develop a general framework, compensating for the ignored characteristics among the node-induced structures, to improve the graph expressivity over the prevalent messagepassing GNNs for various graph downstream tasks, e.g., graph, node and link predictions. Driven by the fact that deep models usually follow the CNA architecture, i.e., a stack of convolution, normalization and activation, where the normalization module generally follows GNNs convolution operations, we accordingly focus on developing a higher-expressivity generalized normalization

