G-CENSOR: GRAPH CONTRASTIVE LEARNING WITH TASK-ORIENTED COUNTERFACTUAL VIEWS

Abstract

Graph Contrastive learning (GCL) has achieved great success in learning representations from unlabeled graph-structure data. However, how to automatically obtain the optimal contrastive views w.r.t specific downstream tasks is little studied. Theoretically, a downstream task can be causally correlated to particular substructures in graphs. The existing GCL methods may fail to enhance model performance on a given task when the task-related semantics are incomplete/preserved in the positive/negative views. To address this problem, we propose G-CENSOR, i.e., Graph Contrastive lEarniNg with taSk-oriented cOunteRfactual views, a modelagnostic framework designed for node property prediction tasks. G-CENSOR can simultaneously generate the optimal task-oriented counterfactual positive/negative views for raw ego-graphs and train graph neural networks (GNNs) with a contrastive objective between the raw ego-graphs and their corresponding counterfactual views. Extensive experiments on eight real-world datasets demonstrate that G-CENSOR can consistently outperform existing state-of-the-art GCL methods to improve the task performance and generalizability of a series of typical GNNs. To the best of our knowledge, this is a pioneer investigation to explore task-oriented graph contrastive learning from a counterfactual perspective in node property prediction tasks. We will release the source code after the review process.

1. INTRODUCTION

Inspired by the convincing success of contrastive learning in the domain of computer vision (Chen et al., 2020; He et al., 2020) and natural language processing (Gao et al., 2021) , graph contrastive learning (GCL) has become an emerging field that extends the idea to graph data (You et al., 2020a; Hassani & Ahmadi, 2020; Zhu et al., 2021; Li et al., 2022) , leading to generalizable, transferable and robust representations from unlabeled graph data (You et al., 2021) . Nevertheless, the generation mechanism of contrastive views, which has been recognized as an essential component in GCL (Zhu et al., 2021; Yin et al., 2022; You et al., 2021) , is still facing the following challenges: (a) Independent of downstream tasks. Although GCL is originally proposed for self-supervised learning, how to obtain the optimal positive view when downstream tasks are available can be an important question Xie et al. (2022) . However, most prior works, whether based on graph diffusion (Hassani & Ahmadi, 2020), uniform sampling (Zhu et al., 2020) , or adaptive sampling (Zhu et al., 2021; You et al., 2021) , ignore the downstream tasks' information. As shown in Figure 1 , whether a generated view is a appropriate positive view depends critically on the downstream tasks Chen et al. ( 2020). (b) Fitting spurious correlations. To introduce task information, learnable data augmentation has been investigated to automatically obtain the positive views for downstream tasks (Yin et al., 2022) . While these techniques have achieved promising performance, they are prone to be plagued by spurious correlations between graph structures and downstream tasks like general supervised methods, thus hurting the generalizability of representation model. (c) Difficulty in negative views selection. Beside positive views, negative sampling is also a vital component in GCL. Contrastive learning can benefit from hard negative samples (Joshua et al., 2021) . Meanwhile, negative samples, actually similar to the raw instances, can lead to a performance drop (Chuang et al., 2020) . Therefore, it can be hard to select suitable negative samples. Some works (He et al., 2020) utilize a great number of negative samples to avoid this trade-off but

