BREAKING THE CURSE OF DIMENSIONALITY FOR PARAMETRIC ELLIPTIC PDES

Abstract

Motivated by recent empirical success, we examine how neural network-based ansatz classes can break the curse of dimensionality for high-dimensional, nonlinear elliptic partial differential equations (PDEs) with variational structure. The high-dimensionality of the PDEs can either be induced through a high-dimensional physical domain or a high-dimensional parameter space. The latter include parametric right-hand sides, parametric domains, and material constants. Our main result shows that any scheme that computes neural network based W 1,p -approximations, leverages the extraordinary approximation capabilities of neural networks and, thus, is able to beat the curse of dimensionality if the ground truth solution is smooth or possesses Barron regularity. Popular examples of W 1,p -convergent schemes include, e.g., the Deep Ritz Method and physics-informed neural networks. We present numerical experiments supporting our theoretical findings.

1. INTRODUCTION

High-dimensional partial differential equations (PDEs) arise naturally in applications with either a high-dimensional domain, a high-dimensional parameter space, or possibly with both. The former includes the Schrödinger equation in quantum physics, the Black-Scholes equation in finance, and the Hamilton-Jacobi-Bellman equation in control theory, we refer to Weinan et al. (2021) ; Bellman (1954) . On the other hand, examples of problems with high-dimensional parameter space are ubiquitous in engineering applications, for instance, in varying material properties, right-hand sides or even in the form of varying computational domains, as discussed in Hennigh et al. (2021); Ohlberger & Rave (2016) . For problems with a high-dimensional physical domain, classical mesh-based approximation schemes face the curse of dimensionality, meaning that the computational cost increases exponentially with the dimension of the problem. In the case of parametric problems, one is typically interested in querying the PDE solution for many different parameter instances, possibly with low inference time. To this end, classical methods need to repeatedly solve the equations for every required parameter instance, a potentially prohibitively expensive or slow computational task, see Biegler et al. (2007) . Even assuming additional, favorable structure of the solution of a high-dimensional PDE -may it be a latent low-dimensionality of the solution or a high degree of smoothness -it remains a challenge for classical methods to approximate the solution with an acceptable accuracy, especially in situations of non-linear solution manifolds as discussed in Ohlberger & Rave (2016); Lee & Carlberg (2020). Artificial neural networks have shown great potential in the approximation of high-dimensional functions, among those computer vision, classification and natural language processing tasks and are known to possess extraordinary approximation capabilities with the possibility to achieve dimensionindependent approximation rates for certain function classes, see Ma et al. (2022) ; Barron (1993) ; Yarotsky ( 2017 Of the aforementioned contributions, the theoretical works either focus on approximation theoretic results or consider linear problems without parametric dependencies. We discuss the relation to our contribution in detail in Section 1.1. The approximation theoretic results guarantee the existence of a neural network with desirable approximation rates but provide no practical way to compute the neural network. For non-linear, parametric PDEs with p-structure, our results deliver the necessary PDE analysis -in a setting suitable for neural network ansatz functions -to alleviate this problem. Instead of explicitly constructing a neural network, we show that it suffices to find a neural network approximation that is close "in energy" to the ground truth solution. This makes a significant difference: Energy approximations can be found by using the variational energy as a loss function or more generally by any W 1,p -convergent approximation scheme -a natural property of a reasonable approximation algorithm. To summarize, our main contributions are the following: • We show that every energy convergent approximation scheme for the p-Dirichlet energy utilizing neural network ansatz functions can leverage the extraordinary approximation capabilities of neural networks. Further, we explain which assumptions on the ground truth solution allow neural networks to beat the curse of dimensionality. We extend these results to parametric problems, where the neural network approximates simultaneously in the physical and the parameter space. Our contributions are the first quantitative error estimates for parametric and non-linear elliptic PDEs for neural network approximation schemes. • From a mathematical point of view, the analysis of non-linear ansatz classes is novel in the case of the p-Laplacian. Existing literature exclusively exploits strategies based on optimality conditions (Galerkin orthogonality) only available for linear ansatz classes, hence, excludes neural networks. Further, to the best of the author's knowledge, error estimates for parametric problems have not been considered in the existing literature.

1.1. MAIN RESULT AND RELATED WORK

For clarity, we present our main result for the case of a parametric right-hand side and with homogeneous Neumann boundary conditions. However, different boundary conditions and parametric dependencies are covered by our analysis. We explain this in the Appendix and refer to Section C. Consider a physical domain Ω ⊆ R dΩ , d Ω ∈ N, and a parameter space P ⊆ R d P , d P ∈ N. Further, let p ∈ (1, ∞) be fixed and denote by (f (τ , •)) τ ∈P a parametric family of right-hand sides. We study the non-linear p-Laplace problem as a prototypical, non-linear elliptic PDE. More precisely, we want to find u * : P × Ω → R satisfying -div |∇ x u * (τ , x)| p-2 ∇ x u * (τ , x) = f (τ , x) for a.e. (τ , x) ⊤ ∈ P × Ω , subjected to -for simplicity -homogeneous Neumann boundary conditions. The case p = 2 retrieves the classical Poisson equation. In this example, the parametric dependencies are induced through the right-hand side and both Ω and P may be high-dimensional. Then, we seek a neural network u θ with input (τ , x) ⊤ ∈ R d P × R dΩ that approximates the solution u * : P × Ω → R simultaneously in the physical domain Ω and the parameter space P. Essential for the statement of our result is the reformulation of equation equation 1 as a minimization problem. We find u * ∈ L p (P, W 1,p (Ω)) as a minimizer of E : L p (P, W 1,p (Ω)) → R, for every v ∈ L p (P, W 1,p (Ω)) defined by E(v) := ˆP 1 p ˆΩ |∇ x v(τ , x)| p dx -ˆΩ f (τ , x) v(τ , x) dx dτ . Then, our main result is the following. (3)



); Gühring & Raslan (2021); Gühring et al. (2020). Therefore, investigating artificial neural networks as ansatz classes for the solution of PDEs or PDE solution operators has recently gained increased interest for high-dimensional and parametric problems. We refer to Kutyniok et al. (2022); Weinan & Wojtowytsch (2022); Jentzen et al. (2021); Chen et al. (2021) for theoretical studies. Successful empirical results of neural network-based applications to PDEs posed in high-dimensional spaces include Hermann et al. (2020); Yu & E (2018); Han et al. (2018); Sirignano & Spiliopoulos (2018). For the parametric setting, we direct the reader to Li et al. (2021); Khoo et al. (2021); Lee & Carlberg (2020); Geist et al. (2021).

Let Ω ⊆ R dΩ , d Ω ∈ N, be a bounded Lipschitz domain and P ⊆ R d P , d P ∈ N, an open set. Moreover, let f ∈ L p ′ (P × Ω), p ∈ (1, ∞), be such that ´Ω f (τ , •) dx = 0 for a.e. τ ∈ P. Denote by u * ∈ L p (P, W 1,p (Ω)), a weak solution of the parametric p-Laplace problem with homogeneous Neumann boundary conditions, i.e., -div(|∇ x u * | p-2 ∇ x u * ) = f in P × Ω , ∂ n u * = 0 on P × ∂Ω .

