LOOK IN THE MIRROR: MOLECULAR GRAPH CON-TRASTIVE LEARNING WITH LINE GRAPH Anonymous

Abstract

Trapped by the label scarcity in molecular property prediction and drug design, graph contrastive learning came forward. A general contrastive model consists of a view generator, view encoder, and contrastive loss, in which the view mainly controls the encoded information underlying input graphs. Leading contrastive learning works show two kinds of view generators, that is, random or learnable data corruption and domain knowledge incorporation. While effective, the two ways also lead to molecular semantics altering and limited generalization capability, respectively. Thus, a decent view that can fully retain molecular semantics and is free from profound domain knowledge is supposed to come forward. To this end, we relate molecular graph contrastive learning with the line graph and propose a novel method termed LGCL. Specifically, by contrasting the given graph with the corresponding line graph, the graph encoder can freely encode the molecular semantics without omission. While considering the information inconsistency and over-smoothing derived from the learning process because of the mismatched pace of message passing in two kinds of graphs, we present a new patch with edge attribute fusion and two local contrastive losses for performance fixing. Compared with state-of-the-art (SOTA) methods for view generation, superior performance on molecular property prediction suggests the effectiveness of line graphs severing as the contrasting views.

1. INTRODUCTION

A deep understanding of molecular properties plays a vital role in the chemical and pharmaceutical domains. In order to computationally discover novel materials and drugs, the molecules will be abstractly regarded as graphs, in which atoms are vertices and bonds are edges Gilmer et al. (2017) ; Goh et al. (2017) ; Chen et al. (2018a) . Thus, the marriage between molecular property prediction and graph learning captured a bunch of researchers and showed their happiness in several fields Yang et al. (2019); Song et al. (2020); Chen et al. (2021); Wu et al. (2022a) . However, this relationship faces the challenges of label scarcity, as deep learning methods are known to consume massive amounts of labeled data, and annotated data are often of limited size and hard to acquire when considering many specific domains. In addition, given the immense differentiation among chemical molecules, existing supervised models could be barely reused in unseen cases Hu et al. (2020); Rong et al. (2020) . Therefore, there are increasing demands for molecular representation learning in an unsupervised or self-supervised manner. Analogously, everything comes with a price. Inspecting the generated views in previous molecular graph contrastive learning unveils two intrinsic limitations. First, data augmentation-based methods adopting random or learnable corruption (e.g., node/edge dropping and graph generation) would lead to inevitable variance in the crucial semantics and further misguide the contrastive learning While effective, they are stinted to the profound domain knowledge that is unfriendly to researchers without such knowledge, thus limiting their generalization capability in other domains. In this context, we are seeking for a decent view that will not be bothered by prefabricated domain knowledge and can maintain the molecular semantic information integrally. Fortunately, we met the line graph, also known as congruent graph in graph theory Whitney (1932) ; Harary & Norman (1960); Jung (1966) . In a line graph, the nodes correspond to the edges of the original graph, and the edges refer to the common nodes of the pair edges in the original graph. In particular, the isomorphism of two line graphs is judged to be consistent with the corresponding two original graphs Whitney (1932); Jung (1966) , which ensures the congruent semantic structure after line graph transformation. In light of the line graph, we propose a method termed LGCL to tackle our expectations. The framework of LGCL is shown in Figure 1 . Specifically, to fill the framework demanding two views, all input molecular graphs are transformed into the corresponding line graph. On such a basis, LGCL equips with a dual-helix graph encoder to learn the hidden representation of two views. Note that, due to the different pace of message passing in the original graph and the corresponding line graph, two issues derive from the learning process, that is, information inconsistency and oversmoothing. For information consistency, we further update the graph encoder with edge attribute fusion to bridge the edge attributes between the two kinds of graphs. Over-smoothing is addressed by a novel intra-local contrastive loss based on the idea of NT-Xent loss; put differently, the intralocal contrastive loss aims to maximize the consense between the edge pairs in the two corresponding views and minimize the consense between different edge pairs within the same views. Moreover, we further give an inter-local contrastive loss to enhance the representation learning. The effectiveness of LGCL is verified under the ubiquitous setting of transfer learning for molecular property prediction Hu et al. (2020) . Through pre-training on two million molecular graphs from ZINC15, LGCL shows superior performance on six out of eight benchmarks for molecular property prediction and acquires the highest position on both average ROC-AUC and average ranking. Additionally, we delve deeper into the proposed components via analytical experiments to further assess their benefits. The contributions are elaborated below: • To the best of our knowledge, we are the first to figure out a way to freely and fully excavate molecular semantics within graph contrastive learning. • Inspired by the line graph, we present an approach, termed LGCL, to tackle our expectations, in which edge attribute fusion and two local contrastive losses are united to address the concomitant issues and enhance molecular representation learning.



Plenty of works have attempted to learn molecule representations discarding the supervision of labels, like graph context prediction Liu et al. (2019), graph-level motif prediction Rong et al. (2020) and masked attribute prediction Hu et al. (2020). In light of the contrastive learning from computer vision, researchers go one step further to model molecules in a contrastive manner with data augmentations You et al. (2020); Suresh et al. (2021). Considering the inherent characteristics of chemical molecules, graph contrastive learning incorporating well-designed domain knowledge has also shown excellent capacity in molecular properties prediction Sun et al. (2021); Fang et al. (2022).

Figure 1: Framework overview of LGCL. Contrasted views consist of the original graph and the corresponding line graph. Input graphs are encoded by a dual-helix graph encoder with edge attribute fusion for information consistency. The whole model is jointly optimized via minimizing the NT-Xent loss and the two local contrastive losses.

