SIMPLE SPECTRAL GRAPH CONVOLUTION FROM AN OPTIMIZATION PERSPECTIVE

Abstract

Recent studies on SGC, PageRank and S 2 GC have demonstrated that several graph diffusion techniques are straightforward, quick, and effective for tasks in the graph domain like node classification. Even though these techniques do not even need labels, they can nevertheless produce more discriminating features than raw attributes for downstream tasks with different classifiers. These methods are data-independent and thus primarily rely on some empirical parameters on polynomial bases (e.g., Monomial and Chebyshev), which ignore the homophily of graphs and the attribute distribution. They are more insensitive to heterophilous graphs due to the low-pass filtering. Although there are many approaches focusing on GNNs based on heterophilous graphs, these approaches are dependent on label information to learn model parameters. In this paper, we study the question: are labels a necessity for GNNs with heterophilous graphs? Based on this question, we propose a framework of self-representation on graphs related to the Least Squares problem. Specifically, we use Generalized Minimum RESidual (GMRES) method, which finds the least squares solution over Krylov subspaces. In theoretical analysis, without label information, we enjoy better features with graph convolution. The proposed method, like previous data-independent methods, is not a deep model and is, therefore, quick, scalable, and simple. We also show performance guarantees for models on real and synthetic data. On a benchmark of real-world datasets, empirically, our method is competitive with existing deep models for node classification.

1. INTRODUCTION

With the development of deep learning, CNNs have been widely used in different applications. A convolutional neural network (CNN) is exploits the shift-invariance, local connectivity, and compositionality of image data. As a result, CNNs extract meaningful local features for various imagerelated problems. Although CNNs effectively capture hidden patterns on the Euclidean grid, there is an increasing number of applications where data is represented in the form of non-Euclidean grid, e.g. in the graph domain. GNNs redefine the convolution on the graph in two different ways: spatial and spectral. Spatialbased methods decompose the convolution operation into an aggregation function and a transformation function. The aggregation function is used to aggregate neighbourhood node information by the mean function, which is somewhat similar to the box filter in traditional image processing. Some representative methods in this category are Message Passing Neural Networks (MPNN (Gilmer et al., 2017)), GraphSAGE (Hamilton et al., 2017) , GAT (Veličković et al., 2017) , etc. Spectral methods are based on Graph Fourier Transformation (GFT). They try to learn a filtering function on the eigenvalues (or graph kernel, heat kernel, etc.) These methods usually use approximations in order to simplify the amount of computation, e.g. Chebyshev polynomials and Monomial polynomials are used by ChebNet (Defferrard et al., 2016) ), GDC (Klicpera et al., 2019) , SGC (Wu et al., 2019) , S 2 GC (Zhu & Koniusz, 2021) . Although spatial and spectral methods effectively extend the convolution operator to the graph domain, they usually suffer from oversmoothing on heterophily graph because they follow the homophily assumption, thus severely affect the node classification task as shown in Figure 1 . 1

