BI-LEVEL PHYSICS-INFORMED NEURAL NETWORKS FOR PDE CONSTRAINED OPTIMIZATION USING BROYDEN'S HYPERGRADIENTS

Abstract

Deep learning based approaches like Physics-informed neural networks (PINNs) and DeepONets have shown promise on solving PDE constrained optimization (PDECO) problems. However, existing methods are insufficient to handle those PDE constraints that have a complicated or nonlinear dependency on optimization targets. In this paper, we present a novel bi-level optimization framework to resolve the challenge by decoupling the optimization of the targets and constraints. For the inner loop optimization, we adopt PINNs to solve the PDE constraints only. For the outer loop, we design a novel method by using Broyden's method based on the Implicit Function Theorem (IFT), which is efficient and accurate for approximating hypergradients. We further present theoretical explanations and error analysis of the hypergradients computation. Extensive experiments on multiple large-scale and nonlinear PDE constrained optimization problems demonstrate that our method achieves state-of-the-art results compared with strong baselines.

1. INTRODUCTION

PDE constrained optimization (PDECO) aims at optimizing the performance of a physical system constrained by partial differential equations (PDEs) with desired properties. It is a fundamental task in numerous areas of science (Chakrabarty & Hanson, 2005; Ng & Dubljevic, 2012) and engineering (Hicks & Henne, 1978; Chen et al., 2009) , with a wide range of important applications including image denoising in computer vision (De los Reyes & Schönlieb, 2013) , design of aircraft wings in aerodynamics (Hicks & Henne, 1978) , and drug delivery (Chakrabarty & Hanson, 2005) in biology etc. These problem have numerous inherent challenges due to the diversity and complexity of physical constraints and practical problems. Traditional numerical methods like adjoint methods (Herzog & Kunisch, 2010) based on finite element methods (FEMs) (Zienkiewicz et al., 2005) have been studied for decades. They could be divided into continuous and discretized adjoint methods (Mitusch et al., 2019 ). The former one requires complex handcraft derivation of adjoint PDEs and the latter one is more flexible and more frequently used. However, the computational cost of FEMs grows quadratically to cubically (Xue et al., 2020) w.r.t mesh sizes. Thus compared with other constrained optimization problems, it is much more expensive or even intractable to solve high dimensional PDECO problems with a large search space or mesh size. To mitigate this problem, neural network methods like DeepONet (Lu et al., 2019) have been proposed as surrogate models of FEMs recently. DeepONet learns a mapping from control (decision) variables to solutions of PDEs and further replaces PDE constraints with the operator network. But these methods require pretraining a large operator network which is non-trivial and inefficient. Moreover,

