DEEP LEARNING SOLUTION OF THE EIGENVALUE PROBLEM FOR DIFFERENTIAL OPERATORS Anonymous

Abstract

Solving the eigenvalue problem for differential operators is a common problem in many scientific fields. Classical numerical methods rely on intricate domain discretization, and yield non-analytic or non-smooth approximations. We introduce a novel Neural Network (NN)-based solver for the eigenvalue problem of differential self-adjoint operators where the eigenpairs are learned in an unsupervised end-to-end fashion. We propose three different training procedures, for solving increasingly challenging tasks towards the general eigenvalue problem. The proposed solver is able to find the M smallest eigenpairs for a general differential operator. We demonstrate the method on the Laplacian operator which is of particular interest in image processing, computer vision, shape analysis among many other applications. Unlike other numerical methods such as finite differences, the partial derivatives of the network approximation of the eigenfunction can be analytically calculated to any order. Therefore, the proposed framework enables the solution of higher order operators and on free shape domain or even on a manifold. Non-linear operators can be investigated by this approach as well.

1. INTRODUCTION

Eigenfunctions and eigenvalues of the Laplacian (among other operators) are important in various applications ranging, inter alia, from image processing to computer vision, shape analysis and quantum mechanics. It is also of major importance in various engineering applications where resonance is crucial for design and safety [Benouhiba & Belyacine (2013) ]. Laplacian eigenfunctions allow us to perform spectral analysis of data measured at more general domains or even on graphs and networks [Shi & Malik (2000) ]. Additionally, the M -smallest eigenvalues of the Laplace-Beltrami operator are fundamental features for comparing geometric objects such as 3D shapes, images or point clouds via the functional maps method in statistical shape analysis [Ovsjanikov et al. (2012) ]. Moreover, in quantum mechanics, the smallest eigenvalues and eigenfunction of the Hamiltonian are of great physical significance [Han et al. (2019) ]. In this paper we present a novel numerical method for the computation of these eigenfunctions (efs) and eigenvalues (evs), where the efs are parameterized by NNs with continuous activation functions, and the evs are directly calculated via the Rayleigh quotient. The resulting efs are therefore smooth functions defined in a parametric way. This is in contrast to the finite element [Pradhan & Chakraverty (2019) ] and finite difference [Saad (2005) ; Knyazev (2000) ] methods in which the efs are defined on either a grid or as piecewise linear/polynomial functions with limited smoothness. In these matrix-based approaches one has to discretize first the problem and to represent it as an eigenvalue problem for a matrix. This in itself is prone to numerical errors. Following [Bar & Sochen (2019) ], we suggest an unsupervised approach to learn the eigenpairs of a differential operator on a specified domain with boundary conditions, where the network simultaneously approximates the eigenfunctions at every entry x. The method is based on a uniformly distributed point set which is trained to satisfy two fidelity terms of the eigenvalue problem formulated as the L 2 and L ∞ -like norms, boundary conditions, orthogonality constraint and regularization. There are several advantages of the proposed setting: (i) the framework is general in the sense that it can be used for non linear differential operators with high order derivatives as well. (ii) Since we sample the domain with a point cloud, we are not limited to standard domains. The problem can be therefore solved in an arbitrary regular domain. (iii) The framework is generic such that additional constraints and regularizers can be naturally integrated in the cost function. (iv) Unlike previous methods, the suggested framework solves simultaneously multiple eigenpairs. This means that we handle a family of PDEs (one for each eigenvalue and finding the eigenvalues themselves) in one network that solves these multiple PDEs together. The method is applied in 1D and 2D for both known and multiple unknown eigenvalues of the Laplacian operator. Quantitative analysis demonstrates the robustness of the method compared with the classical matrix-based methods.

2. RELATED WORK

Many recent approaches have shown promise in using the power of NNs to approximate solutions of differential equations. Classical methods are often prone to weakness due to the discretization of the domain Ω. In [Bar & Sochen (2019) ], the authors propose a solver for both forward and inverse problems, using NNs to model the solution, and penalizing using both the automatic differentiation, and boundary conditions. In [Raissi et al. (2017) ], a similar approach was taken to solve both continuous and discrete time models. In [Chen et al. ( 2018)], differential equation solvers are used as part of the network architecture, and are shown to enhance the smoothness and convergence of the solutions. In order to properly solve differential equations, a representation that captures highorder derivatives is desired. Recently, [Sitzmann et al. (2020) ] proposed a network architecture that illustrates these requirements using periodic activation functions with the proper initialization. Additionally, [Rippel et al. (2015) ] proposed leveraging Discrete Fourier Transform (DFT) to represent the network in spectral space. The well-known power method and its variants [Eastman & Estep (2007) ] has been the main method for addressing the eigenvalue problem. The method works on specific Linear operators, L : L 2 (R d ) → L 2 (R d ). It is done after the continuous equation is reduced numerically to an eigenpair equation for matrices. This process introduces numerical errors even before the solution of the eigen problem. The usage of the power method for spectral operators on Hilbert Spaces was recently shown inn [Erickson et al. (1995) ]. In [Hait-Fraenkel & Gilboa (2019) ] a modified method for nonlinear differential operators was proposed. Furthermore, most power method variants for operators, converge to a single eigenpair. Finding the M smallest eigenpairs can be both computationally and algorithmically challenging. [Han et al. (2020) ] formulated the eigenvalue problem by the stochastic backward equation using the DMC method, where the loss function optimizes the eigenvalue, eigenfunction and the scaled gradient of the eigenfunction. The loss function consists of L 2 norm of two fidelity terms with additional normalization. The algorithm yields the first eigenpair with an optional second eigenpair given some mild prior estimate of the eigenvalue. In the suggested work, we formulate the eigenvalue problem in a direct setting with flexible number of eigenpairs. Additionally, we use L ∞ norms for fidelity and boundary condition terms to accomplish a strong (pointwise) solution.

3. PRELIMINARIES

Let H be a Hilbert space where the inner product for u, v ∈ H is u, v . Let A ∈ O(H) be an operator. Let A * be the adjoined operator defined by A * u, v = u, Av ∀u, v ∈ H. Then A is said to be self-adjoint if A = A * . We start with a short Lemma on self-adjoint operators [Conway (1985) ]. Lemma 3.1 Let H be a Hilbert space. Let A ∈ O(H) be a self-adjoint operator. Then all eigenvalues of A are real. In this work we focus on self-adjoint operators. An eigenpair of an operator L is defined as: (u, λ) s.t. λ ∈ R, where u is the eigenfunction of L and λ is the corresponding eigenvalue. Let L be a self-adjoint operator L : L 2 (R d ) → L 2 (R k ). Our objective is to search for eigenpairs {u i , λ i } such that Lu i + λ i u i = 0 ∀i. (1)



Advanced methods addressing the eigenvalue problem via deep networks were recently introduced. These methods are based on variational Monte Carlo (VMC) and diffusion Monte Carlo (DMC) methods. VMC relies on leveraging physical knowledge to propose an ansatz of the eigenfunction and incorporates the essential physics [Han et al. (2019); Hermann et al. (2019); Pfau et al. (2019); Choo et al. (2019)]. Recently,

