GRAPH NEURAL NETWORKS AS MULTI-VIEW LEARN-ING

Abstract

Graph Neural Networks (GNNs) have demonstrated powerful representation capability in semi-supervised node classification. In this task, there are often three types of information -graph structure, node features, and node labels. Existing GNNs usually leverage both node features and graph structure by feature transformation and aggregation, following end-to-end training via node labels. In this paper, we change our perspective by considering these three types of information as three views of nodes. This perspective motivates us to design a new GNN framework as multi-view learning which enables alternating optimization training instead of end-to-end training, resulting in significantly improved computation and memory efficiency. Extensive experiments with different settings demonstrate the effectiveness and efficiency of the proposed method.

1. INTRODUCTION

Graph is a fundamental data structure that denotes pairwise relationships between entities in a wide variety of domains (Wu et al., 2019b; Ma & Tang, 2021) . Semi-supervised node classification is one of the most crucial tasks on graphs. Given graph structure, node features, and labels on a part of nodes, this task aims to predict labels of the remaining nodes. In recent years, Graph Neural Networks (GNNs) have proven to be powerful in semi-supervised node classification (Gilmer et al., 2017; Kipf & Welling, 2016; Velickovic et al., 2017) . Existing GNN models provide different architectures to leverage both graph structure and node features. Coupled GNNs, such as GCN (Kipf & Welling, 2016) and GAT (Velickovic et al., 2017) , couple feature transformation and propagation to combine node feature and graph structure in each layer. Decoupled GNNs, such as APPNP (Klicpera et al., 2018) , first transform node features and then propagate the transformed features with graph structure for multiple steps. Meanwhile, there are GNN models such as Graph-MLP (Hu et al., 2021) that extract graph structure as regularization when integrating with node features. Nevertheless, the majority of aforementioned GNNs utilize node labels via the loss function for end-to-end training. In essence, existing GNNs have exploited three types of information to facilitate semi-supervised node classification. This understanding motivates us to change our perspective by considering these three types of information as three views of nodes. Then we can treat the design of GNN models as multi-view learning. The advantages of this new perspective are multi-fold. First, we can follow key steps in multi-view learning methods to design GNNs by investigating (1) how to capture node information from each view and (2) how to fuse information from three views. Such superiority offers us tremendous flexibility to develop new GNN models. Second, multi-view learning has been extensively studied (Xu et al., 2013) and there is a large body of literature that can open new doors for us to advance GNN models. To demonstrate the potential of this new perspective, following a traditional multi-view learning method (Xia et al., 2010) , we introduce a shared latent variable to explore these three views simultaneously in a multi-view learning framework for graph neural networks (MULTIVIEW4GNN). The proposed framework MULTIVIEW4GNN can be conveniently optimized in an alternating way, which remarkably alleviates the computational and memory inefficiency issues of the end-to-end GNNs. Extensive experiments under different settings demonstrate that MULTIVIEW4GNN can achieve comparable or even better performance than the end-to-end trained GNNs especially when the labeling rate is low, but it has significantly better computation and memory efficiency.

