SAMPLE-EFFICIENT MULTI-OBJECTIVE MOLECULAR OPTIMIZATION WITH GFLOWNETS

Abstract

Many crucial scientific problems involve designing novel molecules with desired properties, which can be formulated as an expensive black-box optimization problem over the discrete chemical space. Computational methods have achieved initial success but still struggle with simultaneously optimizing multiple competing properties in a sample-efficient manner. In this work, we propose a multiobjective Bayesian optimization (MOBO) algorithm leveraging the hypernetworkbased GFlowNets (HN-GFN) as an acquisition function optimizer, with the purpose of sampling a diverse batch of candidate molecular graphs from an approximate Pareto front. Using a single preference-conditioned hypernetwork, HN-GFN learns to explore various trade-offs between objectives. Inspired by reinforcement learning, we further propose a hindsight-like off-policy strategy to share highperforming molecules among different preferences in order to speed up learning for HN-GFN. Through synthetic experiments, we illustrate that HN-GFN has adequate capacity to generalize over preferences. Extensive experiments show that our framework outperforms the best baselines by a large margin in terms of hypervolume in various real-world MOBO settings.

1. INTRODUCTION

Designing novel molecular structures with desired properties, also referred to as molecular optimization, is a crucial task with great application potential in scientific fields ranging from drug discovery to material design. Molecular optimization can be naturally formulated as a black-box optimization problem over the discrete chemical space, which is combinatorially large (Polishchuk et al., 2013) . Recent years have witnessed the trend of leveraging computational methods, such as deep generative models (Jin et al., 2018) and combinatorial optimization algorithms (You et al., 2018; Jensen, 2019) , to facilitate the optimization. However, the applicability of most prior approaches in real-world scenarios is hindered by two practical constraints: (i) realistic oracles (e.g., wet-lab experiments and high-fidelity simulations) require substantial costs to synthesize and evaluate molecules (Gao et al., 2022) , and (ii) chemists commonly seek to optimize multiple properties of interest simultaneously (Jin et al., 2020b) . For example, in addition to effectively inhibiting a disease-associated target, an ideal drug is desired to be easily synthesizable and non-toxic. Bayesian optimization (BO) (Jones et al., 1998; Shahriari et al., 2015) provides a sample-efficient framework for globally optimizing expensive black-box functions. The basic idea is to construct a cheap-to-evaluate surrogate model, typically a Gaussian Process (GP) (Rasmussen, 2003) , to approximate the true function (also known as the oracle) on the observed dataset. The core objective of BO is to optimize an acquisition function (built upon the surrogate model) in order to obtain informative candidates with high utility for the next round of evaluations. This loop is repeated until the evaluation budget is exhausted. Owing to the fact that a large batch of candidates can be evaluated in parallel in biochemical experiments, we perform batch BO (with large-batch and low-round settings (Angermueller et al., 2020) ) to significantly shorten the entire cycle of optimization. As multi-objective optimization (MOO) problems are prevalent in scientific and engineering applications, MOBO also received broad attention and achieved promising performance by effectively optimizing differentiable acquisition functions (Daulton et al., 2020) . Nevertheless, it is less prominent in discrete problems, especially considering batch settings. The difficulty lies in the fact that no gradients can be leveraged to navigate the discrete space for efficient and effective optimization of 1

