MARS: MARKOV MOLECULAR SAMPLING FOR MULTI-OBJECTIVE DRUG DISCOVERY

Abstract

Searching for novel molecules with desired chemical properties is crucial in drug discovery. Existing work focuses on developing neural models to generate either molecular sequences or chemical graphs. However, it remains a big challenge to find novel and diverse compounds satisfying several properties. In this paper, we propose MARS, a method for multi-objective drug molecule discovery. MARS is based on the idea of generating the chemical candidates by iteratively editing fragments of molecular graphs. To search for high-quality candidates, it employs Markov chain Monte Carlo sampling (MCMC) on molecules with an annealing scheme and an adaptive proposal. To further improve sample efficiency, MARS uses a graph neural network (GNN) to represent and select candidate edits, where the GNN is trained on-the-fly with samples from MCMC. Experiments show that MARS achieves state-of-the-art performance in various multi-objective settings where molecular bio-activity, drug-likeness, and synthesizability are considered. Remarkably, in the most challenging setting where all four objectives are simultaneously optimized, our approach outperforms previous methods significantly in comprehensive evaluations. The code is available at https://github.com/yutxie/mars.

1. INTRODUCTION

Drug discovery aims to find chemical compounds with desired target properties, such as high druglikeness (Bickerton et al., 2012, QED) . The problem is also referred to as molecular design, molecular generation, or molecular search. The space of drug-like chemicals is enormous, approximate 10 33 for realistic drugs that could ever be synthesized (Polishchuk et al., 2013) . Therefore it is very challenging to search for high-quality molecules from such a vast space -enumeration would take almost forever. For a particular disease, finding the right candidates targeting specific proteins further complicates the problem. Instead of enumerating or searching from the immense chemical space, recent work utilizes deep generative models to generate candidate molecules directly (Schwalbe-Koda & Gómez-Bombarelli, 2020). However, most prior work focuses on generating molecules concerning a single property such as drug-likeness (QED) or octanol-water partition coefficient (logP) (Jin et al., 2018; You et al., 2018; Popova et al., 2019; Shi et al., 2020; Zang & Wang, 2020) . While in practical settings, typical drug discovery requires consideration of multiple properties jointly (Nicolaou et al., 2012) . For example, to find drug-like molecules that are easy to synthesize and exhibit high biological activity against the target protein. Naturally, multi-objective molecule design is much more challenging than the single-objective scenario (Jin et al., 2020) . This paper studies the problem of multi-objective molecule design for drug discovery. An ideal solution should be efficient and meet the following criteria. C1: It should satisfy multiple properties with high scores; C2: It should produce novel and diverse molecules; C3: Its generation process does not rely on either expert annotated or wet experimental data collected from a biochemistry lab (since it requires tremendous effort and hard to obtain). Existing molecule generation approaches are mainly designed for the single objective setting, and they could not meet all criteria in the setting of multiple objectives. These methods belong to four categories: a) generating candidates from a learned continuous latent space (Gómez-Bombarelli et al., 2018; Jin et al., 2018) , b) through reinforcement learning (You et al., 2018) , c) using an encoder-decoder translation approach (Jin et al., 2019) , or d) optimizing molecular properties through genetic algorithms (Nigam et al., 2020) . Current stateof-the-art multi-objective molecular generation is a rationale-based method (Jin et al., 2020) . In this approach, the authors propose to build molecules by composing multiple extracted rationales, and the model can generate compounds that are simultaneously active to multiple biological targets. However, such an approach will result in quite complex molecules when we have many objectives. This is because different objectives correspond to different rationales, and including all these rationales could lead to large molecules, which may be less drug-like and hard to be synthesized practically. In this paper, we propose MArkov moleculaR Sampling (MARS), a simple yet flexible method for drug discovery. The basic idea is to start from a seed molecule and keep generating candidate molecules by modifying fragments of molecular graphs from previous steps. It meets all the criteria C1-3. In MARS, the molecular design is formulated as an iterative editing procedure with its total objective consisting of multiple property scores (C1). MARS employs the annealed Markov chain Monte Carlo sampling method to search for optimal chemical compounds, which allows for the exploration of chemicals with novel and different fragments (C2). The proposal to modify molecular fragments is represented using graph neural networks (GNNs), whose parameters are adaptively learned. We used message passing neural networks (MPNNs) in practice (Gilmer et al., 2017) , but other GNNs can fit the framework as well. Furthermore, MARS utilizes the sample paths generated on-the-fly to train the proposal network adaptively. Therefore, it does not rely on external annotated data (C3). With such an adaptive learnable proposal, it keeps improving the generation quality throughout the process. We evaluate MARS and four other baselines, one latest method for each of the four method categories. The benchmark includes a variety of multi-objective generation settings. Experiments show that our proposed MARS achieves state-of-the-art performance on five out of six tasks in terms of a comprehensive evaluation consisting of the success rate, novelty, and diversity of the generated molecules. Notably, in the most challenging setting where four objectives -bio-activities to two different targets, drug-likeness, and synthesizability -are simultaneously considered, our method achieves the state-of-the-art result and outperforms existing methods by 77% in the comprehensive evaluation. Our contributions are as follows: • We present MARS, a generic formulation of molecular design using Markov sampling, which can easily accommodate multiple objectives. • We develop an adaptive fragment-editing proposal based on GNN that is learnable on the fly with only samples self-generated and efficient in exploring the chemical space. • Experiments verifies our proposed MARS framework can find novel and diverse bioactive molecules that are both drug-like and highly synthesizable.

2. RELATED WORK

Recent years have witnessed the success of applying deep generative models and molecular graph representation learning in drug discovery (Schwalbe-Koda & Gómez-Bombarelli, 2020; Guo & Zhao, 2020) . Existing approaches for molecular property optimization can be grouped into four categories, including generation with a) Bayesian inference, b) reinforcement learning, c) encoderdecoder translation models, and d) evolutionary and genetic algorithms. The first category is learning continuous latent spaces for molecular sequences or graphs and generating from such spaces using Bayesian optimization (BO) (Gómez-Bombarelli et al., 2018; Jin et al., 2018; Winter et al., 2019) . These methods rely heavily on the quality of latent representations, which imposes huge challenges to the encoders when there are multiple properties to consider. Unlike the first class, other work uses reinforcement learning (RL) to optimize desired objectives directly in the explicit chemical space (De Cao & Kipf, 2018; Popova et al., 2018; You et al., 2018;  

