DECN: EVOLUTION INSPIRED DEEP CONVOLUTION NETWORK FOR BLACK-BOX OPTIMIZATION

Abstract

We design a deep evolution convolution network (DECN) to overcome the poor generalization of an evolutionary algorithm in handling continuous black-box optimization. DECN is composed of two modules: convolution-based reasoning module (CRM) and selection module (SM), to move from hand-designed optimization strategies to learned optimization strategies. CRM produces a population closer to the optimal solution based on the convolution operators, and SM removes poor solutions. We also design a proper loss function to train DECN so as to force the random population to move near the optimal solution. The experimental results on one synthetic case and two real-world cases show the advantages of learned optimization strategies over human-designed black-box optimization baselines. DECN obtains good performance with deep structure but encounters difficulties in training. In addition, DECN is friendly to the acceleration with Graphics Processing Units (GPUs) and runs 102 times faster than unaccelerated EA when evolving 32 populations, each containing 6400 individuals.

1. INTRODUCTION

Optimization has been an old and essential research topic in history; Many tasks in computer vision, machine learning, and natural language processing can be abstracted as optimization problems. Moreover, many of these problems are black-box, such as neural architecture search (Elsken et al., 2019) and hyperparameter optimization (Hutter et al., 2019) . Various approaches, such as Bayesian optimization (Snoek et al., 2012) and evolutionary algorithms (EAs), including genetic algorithms (Jin et al., 2019; Khadka & Tumer, 2018; Zhang & Li, 2007; Such et al., 2017; Stanley et al., 2019) and evolution strategies (ES) (Wierstra et al., 2014; Vicol et al., 2021; Hansen & Ostermeier, 2001; Auger & Hansen, 2005; Salimans et al., 2017) , have been proposed to deal with these problems in the past. The generalization ability of EAs is poor. Faced with a new black-box optimization task, we need experts to redesign the EA's crossover, mutation, and selection operations to maximize its performance on the target task, resulting in a hand-designed EA with big application limitation. Most importantly, due to the limitation of expert knowledge, only little target function information is used to assist the design of EA, which makes it challenging to adapt to the target task. How to automatically design optimization strategies according to new tasks is crucial. EA is a generative optimization model that realizes the generation from a random population to an optimal solution by generating potential solutions and retaining good solutions. The task of automatically designing an optimization strategy is learning how to automatically generate and retain potential solutions. This paper first attempts to develop a deep evolution convolution network (DECN) to learn to exploit structure in the problem of interest so that DECN can automatically move a random population near the optimal solution for different black-box optimization tasks. DECN uses the process of EAs to guide the design of this new learning-to-optimize architecture. Like EAs, we propose two critical components of DECN to generate and select potential solutions: a convolution-based reasoning module (CRM) and a selection module (SM). For CRM, we need to ensure the exchange of information between individuals in the population to achieve the function of generating potential solutions. We design a lattice-like environment organizing the population into the modified convolution operators and then employ mirror padding (Goodfellow et al., 2016) to generate the potential offspring. SM need to update the population to survive the fittest solutions. We design SM based on a pairwise comparison between the offspring and the input population regarding their fitness, implemented by employing the mask operator. Then, we design the evolution module (EM) based on CRM and SM to simulate one generation of EAs. Finally, we build the DECN by stacking several EMs to cope with the first issue. The untrained DECN does not handle the black-box optimization problem well because it needs information about the target black-box function. In order to better optimize the objective task, we need to design a training set containing objective function information and a practical loss function to guide the parameter training of DECN. The characteristics of black-box functions make it difficult for us to obtain their gradient information to assist in the training of DECN. To overcome the second issue, the following questions must be solved: how to design a proper loss function and training dataset. We construct a differentiable surrogate function set of the target black-box function to obtain the information of the target black-box function. However, the optimal population is usually unknown. The designed loss function is to maximize the difference between the initial and output populations to train DECN towards the optimal solution, where the loss function can be optimized by back-propagation. We test the performance of DECN on six standard black-box functions, protein docking problem, and planner mechanic arm problem. Three population-based optimization baselines, Bayesian optimization (Kandasamy et al., 2020) , and a learning-to-optimize method for black-box optimization (Cao et al., 2019) are employed as references. The results indicate that DECN can automatically learn efficient mapping for unconstrained continuous optimization on high-fidelity and low-fidelity training datasets. Finally, to verify that DECN is friendly to Graphics Processing Units (GPUs)' acceleration, we compare the runtime of DECNs on one 1080Ti GPU with the standard EA.

2. RELATED WORK

There are many efforts that can handle black-box optimization, such as Bayesian optimization (Snoek et al., 2012) and EAs (Mitchell, 1998) . Since the object of DECN is population, it has a strong relationship with EA. Meanwhile, DECN is a new learning-to-optimize (L2O) framework. Appendix A.10 shows our detailed motivations. EAs. EAs are inspired by the evolution of species and have provided acceptable performance for black-box optimization. There are two essential parts to EAs: 1) crossover and mutation: how to generate individuals with the potential to approach the optimal solution; 2) selection: how to discard individuals with inferior performance while maintaining the ones with superior performance. In the past decades, many algorithmic components have been designed for different tasks in EAs. The performance of algorithms varies towards various tasks, as different optimization strategies may be required given diverse landscapes. This paper focuses on two critical issues of EAs: 1) Poor generalization ability. Existing methods manually adjust genetic operators' hyperparameters and design the combination between them (Kerschke et al., 2019; Tian et al., 2020) ; However, its crossover, mutation, and selection modules can only be designed manually based on expert knowledge and cannot effectively interact with the environment (function); that is, they cannot change their elements in real-time to adapt to new problems through the feedback of the objective function. 2) The acceleration of EAs using GPUs is a challenging task. The support for multiple subpopulations to evolve simultaneously has paramount significance in practical applications. Besides, many available genetic operators are unfriendly to the GPU acceleration, as GPUs are weak in processing logical operations. DECN overcomes the above issues. It is adapted to different optimization scenarios, based on which DECN automatically forms optimization strategies. L2O. The most related work is about L2O (Chen et al., 2022) . These methods employ the long short-term memory architecture (LSTM) (Chen et al., 2020; Andrychowicz et al., 2016; Chen et al., 2017; Li & Malik, 2016; Wichrowska et al., 2017; Bello et al., 2017) or multilayer perceptron (MLP) (Metz et al., 2019) as the optimizer to achieve point-based optimization (Sun et al., 2018; Vicol et al., 2021; Flennerhag et al., 2021; Li & Malik, 2016) . However, none of the above methods can handle black-box optimization. Swarm-inspired meta-optimizer (Cao et al., 2019) learns in the algorithmic space of both point-based and population-based optimization algorithms. This method does not consider the advantage of EAs and is a model-free method. Existing L2O techniques rarely focus on black-box optimization. Although several efforts like (Cao et al., 2019; Chen et al., 2017) have coped with these problems, they all deal with small-scale problems in the experimental setting. DECN is a

