LATENT OPTIMIZATION VARIATIONAL AUTOENCODER FOR CONDITIONAL MOLECULE GENERATION

Abstract

Variational autoencoder (VAE) is a generation algorithm, consisting of an encoder and a decoder, and the latent variable is used as the input of the decoder. VAE is widely used for image, audio and text generation tasks. In general, the training of VAE is at risk of posterior collapsing especially for long sequential data. To alleviate this, modified evidence lower bounds (ELBOs) were propsed. However, these approaches heuristically control training loss using a hyper-parameter, and are not way to solve the fundamental problem of vanilla VAE. In this paper, we propose a method to insert an optimization step of the latent variable and alternately update the encoder and decoder for maximizing ELBOs. In experiments, we applied the latent optimization VAE (LOVAE) on ZINC dataset, consisting of string representation of molecules, for the inverse molecular design. We showed that the proposed LOVAE is more stable in the training and achieves better performance than vanilla VAE in terms of ELBOs and molecular generation performance.

1. INTRODUCTION

Deep neural networks (DNNs) have demonstrated a dramatic performance improvement in various applications. Text extraction from image recognition, language translation, speech and natural language recognition, and personal identification by fingerprint and iris have already achieved high accuracy (Wu et al., 2016; Devlin et al., 2018; Awad, 2012; Nguyen et al., 2017) . Recently, these applications became successful commercialized products. For the purpose of generation of image, variational autoencoder (VAE) (Kingma & Welling, 2014) , generative adversarial network (GAN) (Goodfellow et al., 2014) , and reversible generative models (Dinh et al., 2015; 2017; Kingma & Dhariwal, 2018) were proposed and showed much progress (Bleicher et al., 2003; Phatak et al., 2009; Grathwhohl et al., 2018) . These generative models were initially studied in image data and showed better performance than previous models. Since then, it has been extended in research area to generate new sentences (Iqbal & Qureshi, 2020) and to discover new drugs (Chen et al., 2018) and materials (Kim et al., 2018a) . Traditional materials research consists of four steps: molecule design, physical or chemical property prediction, molecular synthesis, and experimental evaluation. These steps are repeated until the desired molecular properties of a molecular structure are satisfied. Until now, trial-and-error techniques based on human knowledge have been widely used. However, they are time consuming and very expensive. In order to improve the traditional method, research for a high-throughput computational screening (HTCS) (Bleicher et al., 2003) was conducted. However, this also had limitations such as high computational cost, predefined molecular structures by human knowledge, and low accuracy of simulation. Unlike the traditional approach, inverse molecular design is an attempt to find novel molecules that satisfy desired properties from exploring a large chemical space (Sanchez-Lengeling & Aspuru-Guzik, 2018) . It extracts knowledge of potential molecular structures and properties from accumulated molecular structure databases (PubChem, ZINC, etc.) and proposes new molecular structures that do not exist in their database (Bolton et al., 2008; Irwin et al., 2012) . With the inverse molecular design, it is possible to save cost by conducting molecular synthesis and experimental evaluation only for molecular structures having desired properties instead of searching almost infinite chemical space.

