

Abstract

In many applications of machine learning, like drug discovery and material design, the goal is to generate candidates that simultaneously maximize a set of objectives. As these objectives are often conflicting, there is no single candidate that simultaneously maximizes all objectives, but rather a set of Pareto-optimal candidates where one objective cannot be improved without worsening another. Moreover, in practice, these objectives are often under-specified, making the diversity of candidates a key consideration. The existing multi-objective optimization methods focus predominantly on covering the Pareto front, failing to capture diversity in the space of candidates. Motivated by the success of GFlowNets for generation of diverse candidates in a single objective setting, in this paper we consider Multi-Objective GFlowNets (MOGFNs). MOGFNs consist of a novel Conditional GFlowNet which models a family of single-objective sub-problems derived by decomposing the multi-objective optimization problem. Our work is the first to empirically demonstrate conditional GFlowNets. Through a series of experiments on synthetic and benchmark tasks, we empirically demonstrate that MOGFNs outperform existing methods in terms of Hypervolume, R2-distance and candidate diversity. We also demonstrate the effectiveness of MOGFNs over existing methods in active learning settings. Finally, we supplement our empirical results with a careful analysis of each component of MOGFNs.

1. INTRODUCTION

Decision making in practical applications often involves reasoning about multiple, often conflicting, objectives (Keeney et al., 1993) . For example, in drug discovery, the goal is to generate novel drug-like molecules that inhibit a target, are easy to synthesize and can safely be used by humans (Dara et al., 2021) . Unfortunately, these objectives often conflict -molecules effective against a target might also have adverse effects on humans -so there is no single molecule which maximizes all the objectives simultaneously. Such problems fall under the umbrella of Multi-Objective Optimization (MOO; Ehrgott, 2005; Miettinen, 2012) , wherein one is interested in identifying Pareto-optimal candidates. The set of Pareto-optimal candidates covers all the best tradeoffs among the objectives, i.e., the Pareto front, where each point on that front corresponds to a different set of weights associated with each of the objectives. In-silico drug discovery and material design are typically driven by proxies trained with finite data, which only approximate the problem's true objectives, and therefore include intrinsic epistemic uncertainty associated with their predictions. In such problems, not only it is important to cover the Pareto front, but also to generate sets of diverse candidates at each solution of the front so as to increase the likelihood of success in downstream evaluations (Jain et al., 2022) . Generative Flow Networks (GFlowNets; Bengio et al., 2021a; b) are a recently proposed family of probabilistic models which tackle the problem of diverse candidate generation. Contrary to the reward maximization view of reinforcement learning (RL) and Bayesian optimization (BO), GFlowNets sample candidates with probability proportional to the reward. Sampling candidates, as opposed to greedily generating them, implicitly encourages diversity in the generated candidates. GFlowNets have shown promising results in single objective problems of molecule generation (Bengio et al., 2021a) and biological sequence design (Jain et al., 2022) . In this paper, we study Multi-Objective GFlowNets (MOGFNs), extensions of GFlowNets which tackle the multi-objective optimization problem. We consider two variants of MOGFNs -(a) Preference-Conditional GFlowNets (MOGFN-PC) which combine Reward-Conditional GFlowNets (Bengio et al., 2021b) with Weighted Sum Scalarization (Ehrgott, 2005) and (b) MOGFN-AL, an extension of GFlowNet-AL (Jain et al., 2022) for multi-objective active learning settings. We empirically demonstrate the advantage of MOGFNs over existing approaches on a variety of highdimensional multi-objective optimization tasks: the generation of small molecules, DNA aptamer sequences and fluorescent proteins. Our contributions are as follows: C1 We demonstrate how two variants of GFlowNets -MOGFN-PC and MOGFN-AL -can be applied to multi-objective optimization. Our work is the first successful empirical validation of Reward-Conditional GFlowNets (Bengio et al., 2021b) . C2 Through a series of experiments on molecule generation and sequence generation we demonstrate that MOGFN-PC generates diverse Pareto-optimal candidates. C3 In a challenging active learning task for designing fluorescent proteins, we show that MOGFN-AL results in significant improvements to sample-efficiency and diversity of generated candidates. C4 We perform a thorough analysis of the main components of MOGFNs to provide insights into design choices that affect performance.

2.1. MULTI-OBJECTIVE OPTIMIZATION

Multi-objective optimization (MOO) involves finding a set of feasible candidates x ⋆ ∈ X which all simultaneously maximize a set of objectives: max x∈X (R 1 (x), . . . , R d (x)) . In general, the objectives being optimized can be conflicting such that there is no single x ⋆ which simultaneously maximizes all objectives. Consequently, the concept of Pareto optimality is adopted in MOO, giving rise to a set of solutions trading off the objectives in different ways. Given x 1 , x 2 ∈ X , x 1 is said to dominate x 2 , written (x 1 ≻ x 2 ), iff R i (x 1 ) ≥ R i (x 2 ) ∀i ∈ {1, . . . , d} and ∃k ∈ {1, . . . , d} such that R k (x 1 ) > R k (x 2 ). A candidate x ⋆ is Pareto-optimal if there exists no other solution x ′ ∈ X which dominates x ⋆ . In other words, for a Pareto-optimal candidate it is impossible to improve one objective without sacrificing another. The Pareto set is the set of all Pareto-optimal candidates in X , and the Pareto front is defined as the image of the Pareto set in objective-space. It is important to note that since the objectives being optimized in general might not be injective, any point on the Pareto front can be the image of several candidates in the Pareto set. This introduces a notion of diversity in the candidate space, capturing all the candidates corresponding to a point on the Pareto front, that is critical for applications such as drug discovery. While there are several paradigms for tackling the MOO problem (Ehrgott, 2005; Miettinen, 2012; Pardalos et al., 2017) , we consider Scalarization, where the multi-objective problem is decomposed into simpler single-objective problems, as it is well suited for the GFlowNet formulation introduced in Section 3.1. A set of weights (preferences) ω i are assigned to the objectives R i , such that ω i ≥ 0 and d i=1 ω i = 1. The MOO problem in Equation 1 is then decomposed into solving single-objective sub-problems of the form max x∈X R(x|ω), where R is a scalarization function. Weighted Sum Scalarization, R(x|ω) = d i=1 ω i R i (x) is a widely used scalarization function which results in Pareto optimal candidates for problems with a convex Pareto front (Ehrgott, 2005)  . Weighted Tchebycheff, R(x|ω) = min 1≤i≤d ω i |R i (x) -z ⋆ i | , where z ⋆ i denotes some ideal value for objective R i , results in Pareto optimal solutions even for problems with a non-convex Pareto front (Pardalos et al., 2017) . See Appendix B for more discussion on scalarization. In summary, using scalarization, the MOO problem can be viewed as solving a family of single-objective optimization problems.

2.2. GFLOWNETS

Generative Flow Networks (Bengio et al., 2021a; b) are a family of probabilistic models which generate, through a sequence of steps,compositional objects x ∈ X with probability proportional to a given reward R : X → R + . The sequential construction of x ∈ X can be described as a trajectory

