TOWARDS EQUIVARIANT GRAPH CONTRASTIVE LEARNING VIA CROSS-GRAPH AUGMENTATION

Abstract

Leading graph contrastive learning (GCL) frameworks conform to the invariance mechanism by encouraging insensitivity to different augmented views of the same graph. Despite the promising performance, invariance worsens representation when augmentations cause aggressive semantics shifts. For example, dropping the super-node can dramatically change a social network's topology. In this case, encouraging invariance to the original graph can bring together dissimilar patterns and hurt the task of instance discrimination. To resolve the problem, we get inspiration from equivariant self-supervised learning and propose Equivariant Graph Contrastive Learning (E-GCL) to encourage the sensitivity to global semantic shifts. Viewing each graph as a transformation to others, we ground the equivariance principle as a cross-graph augmentation -graph interpolationto simulate global semantic shifts. Without using annotation, we supervise the representation of cross-graph augmented views by linearly combining the representations of their original samples. This simple but effective equivariance principle empowers E-GCL with the ability of cross-graph discrimination. It shows significant improvements over the state-of-the-art GCL models in unsupervised learning and transfer learning. Further experiments demonstrate E-GCL's generalization to various graph pre-training frameworks.

1. INTRODUCTION

Graph contrastive learning (GCL) (You et al., 2020; Suresh et al., 2021; Xu et al., 2021) is a prevailing paradigm for self-supervised learning (Chen et al., 2020; Zbontar et al., 2021) on graph-structured data. It typically pre-trains a graph neural network (GNN) (Dwivedi et al., 2020) without labeled data, in an effort to learn generalizable representations and boost the fine-tuning on downstream tasks. The common theme across recent GCL studies is instance discrimination (Dosovitskiy et al., 2014; Purushwalkam & Gupta, 2020 ) -viewing each graph as a class of its own, and differing it from other graphs. It galvanizes representation learning to capture discriminative characteristics of graphs. Towards this end, leading GCL works usually employ two key modules: graph augmentation and contrastive learning. Specifically, graph augmentation adopts the "intra-graph" strategy to create multiple augmented views of each graph, such as randomly dropping nodes (You et al., 2020) or adversarially perturbing edges (Suresh et al., 2021) . The views stemming from the same graph constitute the positive samples of this class, while the views of other graphs are treated as negatives. Consequently, contrastive learning encourages the agreement between positive samples and the discrepancy between negatives. This procedure essentially imposes "invariance" (Purushwalkam & Gupta, 2020; Dangovski et al., 2022) upon representations -making the anchor graph's representation invariant to its intra-graph augmentations (Figure 1a ). Formally, let g be the anchor graph, P be the groups of intra-graph augmentations, and ϕ(•) be the GNN encoder. The "invariance to intra-graph augmentations" mechanism states ϕ(g) = ϕ(T p (g)), ∀p ∈ P -the representation ϕ(g) is insensitive to the changes in augmentation p, where T p (g) is the action of augmentation p on graph g. We refer to works adopting this mechanism as Invariant Graph Contrastive Learning (I-GCL). However, we argue that invariance to intra-graph augmentations alone is insufficient to improve the semantic quality of graph representations and boost the downstream performance:

availability

//anonymous.4open.science/

