LOW-RANK GRAPH NEURAL NETWORKS INSPIRED BY THE WEAK-BALANCE THEORY IN SOCIAL NETWORKS Anonymous

Abstract

Graph Neural Networks (GNNs) have achieved state-of-the-art performance on node classification tasks by exploiting both the graph structures and node features. Generally, most existing GNNs depend on the implicit homophily assumption that nodes belonging to the same class are more likely to be connected. However, GNNs may fail to model heterophilious graphs where nodes with different labels tend to be linked, as shown in recent studies. To address this issue, we propose a generic GNN applicable to both homophilious and heterophilious graphs, namely Low-Rank Graph Neural Network (LRGNN). In detail, we aim at computing a coefficient matrix such that the sign of each coefficient reveals whether the corresponding two nodes belong to the same class, which is similar to the sign inference problem in Signed Social Networks. In this paper, we show that signed graphs are naturally generalized weakly-balanced for node classification tasks. Motivated by this observation, we propose to leverage low-rank matrix factorization (LRMF) to recover a coefficient matrix from a partially observed signed adjacency matrix. To effectively capture the node similarity, we further incorporate the low-rank representation (LRR) method. Our theoretical result shows that under the update rule of node representations, LRR obtained by solving a subspace clustering problem can recover the subspace structure of node representations. To solve the corresponding optimization problem, we utilize an iterative optimization algorithm with a convergence guarantee and develop a neural-style initialization manner that enables fast convergence. Finally, extensive experimental evaluation on both real-world and synthetic graphs has validated the superior performance of LRGNN over various state-of-the-art GNNs. In particular, LRGNN can offer clear performance gains in a scenario when the node features are not informative enough.

1. INTRODUCTION

Graphs (or networks) are ubiquitous in a variety of fields, such as social networks, biology, and chemistry. Many real-world networks follow the Homophily assumption, i.e., linked nodes tend to share the same label or have similar features; while for graphs with heterophily, nodes with different labels are more likely to form a link. For example, many people tend to connect with people of the opposite sex in dating graphs. For graphs with homophily, Graph Neural Networks (GNNs) variants (Kipf & Welling, 2017; Hamilton et al., 2017; Velickovic et al., 2018) have achieved remarkable successes on various graph mining tasks. Among them, Graph Convolutional Network (GCN) (Kipf & Welling, 2017) and Graph Attention Networks (GATs) (Velickovic et al., 2018) are representative methods. However, the performance of GNNs deteriorates when learning on graphs with heterophily, in that the smoothing operation used in traditional GNNs tends to make representations of neighboring nodes similar, even though they have different labels. Some designs (Zhu et al., 2020; Chien et al., 2021; Lim et al., 2021) have been proposed to enhance the representational power of GNNs under heterophilous scenarios (see Zheng et al. (2022) for a survey). Among them, high-pass filters are the most frequently used components since they can push away a node from its neighbors in the embedding space, which conforms to the characteristic of heterophily that nodes are generally dissimilar to their neighbors. High-pass filters are usually realized by negating the normalized adjacency matrix. In the spatial graph convolution domain, signed message passing (Yan et al., 2021; Bo et al., 2021) allows negative aggregation coefficients so as to push away those neighboring heterophilious nodes. However, most existing methods have weaknesses that restrict their representational power. Spectral-based methods (Chien et al., 2021; Luan et al., 2021) combine high-pass filters with low-pass ones by linearly combining the outputs of intermediate layers. These methods fail to capture the node-level homophily ratio as they utilizes only one type of convolutional filters in each layer. Additionally, spatial-based methods (Bo et al., 2021; Yang et al., 2021) update the representation of each node by computing a learnable weighted combination of the representations of its neighbors and updating the aggregation coefficients based on the attention function used in Graph Attention Network (GAT) (Velickovic et al., 2018) . As GAT computes a form of static attention and the ranking of the attention scores is unconditioned on the query node (Brody et al., 2022) , this attention function is prone to produce uniform attention scores and cannot distinguish nodes of different classes when their distributions of features are of small difference (Fountoulakis et al., 2022) . In this paper, we address the challenges of generalizing GNNs to heterophilious graphs with the help of social theory developed from signed social networks (SSNs) whose positive links represent the friendship between two users while whose negative links represent enmity. Similarly, we call the graphs with negative edges as signed graphs. In SSNs, a practical theory called the weak balance theory (Davis, 1967) modifies the structural balance theory (Cartwright & Harary, 1956 ) by eliminating the patter "an enemy of my enemy is my friend", and keeping the following patterns of "an enemy of my friend is my enemy", "a friend of my friend is my friend", and "a friend of my enemy is my enemy". In the context of node classification task, we can view homophily and heterophily as friendship and enmity, respectively. It is easy to verify that signed graphs are weakly balanced when considering node classification tasks, e.g., a homophilious neighbor of my homophilious neighbor is also a homophilious node to me but a heterophilious neighbor of my heterophilious neighbor is not necessarily a homophilious node. The weak balance naturally leads to a global low-rank structure for the network, based on which the sign inference problem can be formulated as a low-rank matrix completion problem (LRMC) (Hsieh et al., 2012) . The weak-balance theory motivates us to apply low-rank approximation approaches to node classification on heterophilious graphs. Specifically, given a partially observed signed adjacency matrix (where negative edges are allowed), we aim to recover a coefficient matrix Z via low-rank approximation methods, such that a positive Z i,j implies node v i and v j have the same label and the magnitude of Z i,j represents the importance of node v j to v i . Then we can use Z to update node representations by performing feature propagation in GNNs. Due to the polynomial time complexity of solving LRMC which is practically infeasible for large networks, we resort to the low-rank matrix factorization (LRMF) technique that is scalable to large graphs. The low-rank approximation of signed networks can achieve satisfactory or even exact recovery under certain assumptions as stated in Davenport & Romberg (2016) . Furthermore, to better capture the similarity between node representations, we leverage the low-rank representation(LRR) (Liu et al., 2010) learning for recovering the underlying subspace structure on heterophilious graphs. Thus, we name the new derived GNN as Low Rank Graph Neural Network (LRGNN). It is important to note that the low-rank assumption was used to improve the defense over adversarial examples (Jin et al., 2020) , it is not designed for heterophilious graph modeling and not implemented for signed graphs. To solve the corresponding non-convex optimization problem, we utilize the softImpute-ALS algorithm (Hastie et al., 2015) which minimizes the objective function by minimizing a surrogate function. Though LRGNN characterizes a node's representation using the linear combination of all the node representations, we can reduce the time complexity to linear by leveraging some tricks of matrix multiplication. Extensive experimental results on both real-world and synthetic datasets showed the superior performance and efficiency of LRGNN over state-of-the-art methods. We put all proofs and the discussion of related work in Appendix due to the space limitation.

2. PRELIMINARIES

Notations. Denote by G = (V, E) a undirected graph, where V and E denote the node set and edge set, respectively. The nodes are described by a node feature matrix X ∈ R n×f , where n and f are the number of nodes and number of features per node, respectively. Y ∈ R n×c is the node label matrix. The neighbor set of node v i is denoted by N i . We denote the node representation matrix in the l-th layer by H (l) . We let A denote the adjacency matrix that generally A i,j = 1 if (i, j) ∈ E and 0 otherwise; while in signed graphs (networks), we extend the values of adjacency matrix to

