WIDE GRAPH NEURAL NETWORK

Abstract

Graph Neural Networks from the spatial and the spectral domains often suffer from the following problems: over-smoothing, poor flexibility, and low performance on heterophily. In this paper, we provide a unified view of GNNs from the matrix space analysis perspective to identify potential reasons for these problems and propose a new GNN framework to address them, called Wide Graph Neural Network (WGNN). We formulate GNNs as two components: one is for constructing a non-parametric feature space, and the other is for learning the parameters to re-weight the feature space. For instance, spatial GNNs encode the adjacency matrix multiplication as the feature space and stack layers to re-weight it, and spectral ones sum the polynomials to build the feature space and learn shared model weights. Instead, WGNN constructs the space by concatenating all polynomials and re-weights them individually. This mechanism reduces the unnecessary constraints on the feature space due to the concatenation, which avoids over-smoothing and allows independent parameters for better flexibility. Beyond the parameter independence property, WGNN enjoys further flexibility in adding matrices with arbitrary columns. For instance, by taking the principal components of the adjacency matrix, we can significantly improve the representation of heterophilic graphs. We provide a detailed theoretical analysis and conduct extensive experiments on eight datasets to show the superiority of the proposed WGNN. 1

1. INTRODUCTION

𝑊 (") 𝐴 # 𝑊 ($) 𝐴 # 𝑊 (%) 𝐴 # 𝑋 𝐻 𝑋 𝑃"(𝐿 ') 𝑃$(𝐿 ') 𝑃%(𝐿 ') 𝑊 (") 𝛾 (") 𝐻 𝐻 = 𝐴 %! 𝑋 & 𝑊 (#) % #&' 𝐻 = ( 𝑃 # 𝐿 + 𝑋𝑊 (#) % #&' + ( 𝑆 ( 𝑊 (() ( 𝐻 = ( 𝛾 # 𝑃 # 𝐿 + 𝑋𝑊 (') % #&' Spatial GNN Spectral GNN WGNN 𝑃"(𝐿 ') 𝑃$(𝐿 ') 𝑃%(𝐿 ') 𝑆& 𝑋 𝑊 (") 𝑊 ($) 𝑊 (%) 𝑊 (') 𝐻 Parameters Data space 𝛾 ($) 𝛾 (%) ⋯ Figure 1: WGNN compared with current GNNs Graph neural networks (GNNs) have demonstrated their great potential in representation learning for graph-structured data, such as social networks, transportation networks, protein interaction networks, and chemical structures (Fan et al., 2019; Wu et al., 2020; Zheng et al., 2022) . Despite the success, existing GNNs still suffer some issues in the following. Firstly, the spatial GNNs aggregate the information from the connected nodes, resulting in the well-known over-smoothing (Cai & Wang, 2020) . Secondly, the spatial models assume that the features of connected nodes are similar; however, this assumption does not hold in heterophilic graphs (Zheng et al., 2022) . Thirdly, the spectral GNNs use polynomials to approach arbitrary graph filters (He et al., 2021; Klicpera et al., 2019; Defferrard et al., 2016) . In the absence of layer stacking, the spectral GNNs are exempt from the issue of over-smoothing. However, these spectral GNNs still perform poorly on heterophilic graphs since each polynomial term also shares the same assumption of similarity in neighbors. In addition, spectral methods share the parameters for each polynomial term, leading to a less flexible architecture. To better understand the problems in both spatial and spectral domains, efforts exist that integrate GNNs, e.g., from the perspective of optimization objectives (Ma et al., 2021; Zhu et al., 2021) . However, they focus on summarizing general formulas while lacking a clear explanation of the problems.



The implementation of WGNN is available at https://drive.google.com/drive/folders/ 1A6VWiPmKRhCNfdcuFJvnxTiTgzgbJIZ6?usp=sharing 1

