LINK PREDICTION WITHOUT GRAPH NEURAL NET-WORKS

Abstract

Link prediction, which consists of predicting edges based on graph features, is a fundamental task in many graph applications. As for several related problems, Graph Neural Networks (GNNs), which are based on an attribute-centric messagepassing paradigm, have become the predominant framework for link prediction. GNNs have consistently outperformed traditional topology-based heuristics, but what contributes to their performance? Are there simpler approaches that achieve comparable or better results? To answer these questions, we first identify important limitations in how GNN-based link prediction methods handle the intrinsic class imbalance of the problem-due to the graph sparsity-in their training and evaluation. Moreover, we propose Gelato, a novel topology-centric framework that applies a topological heuristic to a graph enhanced by attribute information via graph learning. Our model is trained end-to-end with an N-pair loss on an unbiased training set to address class imbalance. Experiments show that Gelato is 145% more accurate, trains 11 times faster, infers 6,000 times faster, and has less than half of the trainable parameters compared to state-of-the-art GNNs for link prediction.

1. INTRODUCTION

Machine learning on graphs supports various structured-data applications including social network analysis (Tang et al., 2008; Li et al., 2017; Qiu et al., 2018b) , recommender systems (Jamali & Ester, 2009; Monti et al., 2017; Wang et al., 2019a) , natural language processing (Sun et al., 2018a; Sahu et al., 2019; Yao et al., 2019) , and physics modeling (Sanchez-Gonzalez et al., 2018; Ivanovic & Pavone, 2019; da Silva et al., 2020) . Among the graph-related tasks, one could argue that link prediction (Lü & Zhou, 2011; Martínez et al., 2016) is the most fundamental one. This is because link prediction not only has many concrete applications (Qi et al., 2006; Liben-Nowell & Kleinberg, 2007; Koren et al., 2009) but can also be considered an (implicit or explicit) step of the graph-based machine learning pipeline (Martin et al., 2016; Bahulkar et al., 2018; Wilder et al., 2019 )-as the observed graph is usually noisy and/or incomplete. In recent years, Graph Neural Networks (GNNs) (Kipf & Welling, 2017; Hamilton et al., 2017; Veličković et al., 2018) have emerged as the predominant paradigm for machine learning on graphs. Similar to their great success in node classification (Klicpera et al., 2018; Wu et al., 2019; Zheng et al., 2020) and graph classification (Ying et al., 2018; Zhang et al., 2018a; Morris et al., 2019) , GNNs have been shown to achieve state-of-the-art link prediction performance (Zhang & Chen, 2018; Yun et al., 2021; Pan et al., 2022) . Compared to classical approaches that rely on expert-designed heuristics to extract topological information (e.g., Common Neighbors (Newman, 2001) , Adamic-Adar (Adamic & Adar, 2003 ), Preferential Attachment (Barabási et al., 2002) ), GNNs have the potential to discover new heuristics via supervised learning and the natural advantage of incorporating node attributes. However, there is little understanding of what factors contribute to the success of GNNs in link prediction, and whether simpler alternatives can achieve comparable performance-as recently found for node classification (Huang et al., 2020) . GNN-based methods approach link prediction as a binary classification problem. Yet different from other classification problems, link prediction deals with extremely class-imbalanced data due to the sparsity of real-world graphs. We argue that class imbalance should be accounted for in both training and evaluation of link prediction. In addition, GNNs combine topological and attribute information by learning topology-smoothened attributes (embeddings) via message-passing (Li et al., 2018) . This attribute-centric mechanism has been proven 7 1 1 1 7 1 !"# 1 $"1 %&'1 ( ) *221 1 +1 (a) Link prediction for attributed graphs 8 1 1 1 8 1 !"# 1 $ "1 %&'1 () * +221 1 ,1 (b) GNN: topology → attributes 8 1 1 1 8 1 !" 1 #!1 $%&1 '( ) *221 1 +1 (c) Gelato: attributes → topology Figure 1 : GNN incorporates topology into attributes via message-passing, which is effective for tasks on the topology. Link prediction, however, is a task for the topology, which motivates the design of Gelato-a novel framework that leverages graph learning to incorporate attributes into topology. effective for tasks on the topology such as node classification (Ma et al., 2020) , but link prediction is a task for the topology, which naturally motivates topology-centric paradigms (see Figure 1 ). The goal of this paper is to address the key issues raised above. We first show that the evaluation of GNN-based link prediction pictures an overly optimistic view of model performance compared to the (more realistic) imbalanced setting. Class imbalance also prevents the generalization of these models due to bias in their training. Instead, we propose the use of the N-pair loss with an unbiased set of training edges to account for class imbalance. Moreover, we present Gelato, a novel framework that combines topological and attribute information for link prediction. As a simpler alternative to GNNs, our model applies topology-centric graph learning to incorporate node attributes directly into the graph structure, which is given as input to a topological heuristic, Autocovariance, for link prediction. Extensive experiments demonstrate that our model significantly outperforms state-of-the-art GNN-based methods in both accuracy and scalability. To summarize, our contributions are: (1) we scrutinize the training and evaluation of supervised link prediction methods and identify their limitations in handling class imbalance; (2) we propose a simple, effective, and efficient framework to combine topological and attribute information for link prediction without using GNNs; and (3) we introduce an N-pair link prediction loss combined with an unbiased set of training edges that we show to be more effective at addressing class imbalance.

2. LIMITATIONS IN SUPERVISED LINK PREDICTION EVALUATION & TRAINING

Supervised link prediction is often formulated as a binary classification problem, where the positive (or negative) class includes node pairs connected (or not connected) by a link. A key difference between link prediction and typical classification problems (e.g., node classification) is that the two classes in link prediction are extremely imbalanced, since most real-world graphs of interest are sparse (see Table 1 ). However, we find that class imbalance is not properly addressed in both evaluation and training of existing supervised link prediction approaches, as discussed below. Link prediction evaluation. Area Under the Receiver Operating Characteristic Curve (AUC) and Average Precision (AP) are the two most popular evaluation metrics for supervised link prediction (Kipf & Welling, 2016; Zhang & Chen, 2018; Chami et al., 2019; Zhang et al., 2021; Cai et al., 2021; Yan et al., 2021; Zhu et al., 2021; Chen et al., 2022; Pan et al., 2022) . We first argue that, as in other imbalanced classification problems (Saito & Rehmsmeier, 2015) , AUC is not an effective evaluation metric for link prediction as it is biased towards the majority class (non-edges). On the other hand, AP and other rank-based metrics such as Hits@k-used in Open Graph Benchmark (OGB) (Hu et al., 2020) -are effective for imbalanced classification if evaluated on a test set that follows the original class distribution. Yet, existing link prediction methods (Kipf & Welling, 2016; Zhang & Chen, 2018; Cai et al., 2021; Zhu et al., 2021; Pan et al., 2022) compute AP on a test set that contains all positive test pairs and only an equal number of random negative pairs. Similarly, OGB computes Hits@k against a very small subset of random negative pairs. We term these approaches biased testing as they highly overestimate the ratio of positive pairs in the graph. Evaluation metrics based on these biased test sets provide an overly optimistic measurement of the actual performance in unbiased testing, where every negative pair is included in the test set. In fact, in real applications where test positive edges are not known a priori, it is impossible to construct those biased test sets to begin with. Below, we also present an illustrative example of the misleading performance evaluation based on biased testing.

