IMPROVED TRAINING OF PHYSICS-INFORMED NEU-RAL NETWORKS USING ENERGY-BASED PRIORS: A STUDY ON ELECTRICAL IMPEDANCE TOMOGRAPHY

Abstract

Physics-informed neural networks (PINNs) are attracting significant attention for solving partial differential equation (PDE) based inverse problems, including electrical impedance tomography (EIT). EIT is non-linear and especially its inverse problem is highly ill-posed. Therefore, successful training of PINN is extremely sensitive to interplay between different loss terms and hyper-parameters, including the learning rate. In this work, we propose a Bayesian approach through datadriven energy-based model (EBM) as a prior, to improve the overall accuracy and quality of tomographic reconstruction. In particular, the EBM is trained over the possible solutions of the PDEs with different boundary conditions. By imparting such prior onto physics-based training, PINN convergence is expedited by more than ten times faster to the PDE's solution. Evaluation outcome shows that our proposed method is more robust for solving the EIT problem.

1. INTRODUCTION

Physics-informed neural networks (PINNs) (Raissi et al., 2019) parameterize the solution of a partial differential equation (PDE) using a neural network and trains the neural network to predict the solution's scalar value for any given point inside the problem's domain by minimizing the residual PDE and associated boundary conditions (BCs). Various other factors such as gradient stiffness (Wang et al., 2021) and complex parameter settings, lack of constraints and regularization (Krishnapriyan et al., 2021) , often cause major issues in training PINNs and makes them very sensitive to hyperparameters and regularization. For example, depending on the PDE, the interplay between BCs residuals and PDEs residuals can result in invalid solutions that mostly satisfy one type of constraints over the others. In general, we believe that some of these problems are attributable to the lack of joint representation in PINNs, as these models are assumed to implicitly learn that two points in a close vicinity possibly have a similar solution. However, this representation is hard-coded in the numerical methods as they update the solution value of each mesh point based on the neighboring mesh points. In this work, we augment PINNs with a joint representation via a Bayesian approach. We train a data-driven prior over joint solutions of PDEs on the entire domain and boundary points. This prior relates the predictions of PINNs via explicit joint representation and encourage learning a coherent and valid solution. Priors have been studied extensively in statistical approach to solve inverse problems (Kaipio et al., 2000; Ahmad et al., 2019; Abhishek et al., 2022; Strauss & Khan, 2015) and also recently have been used in data-driven approaches as well (Ramzi et al., 2020) . More importantly, we focused on the non-linear and ill-posed electrical impedance tomography (EIT)

annex

inverse problem and experimentally show that using our data-driven Bayesian approach results in accurate, fast, and more robust training algorithms. We have the following contributions:• We present various experimental studies which circumvent instability in training PINNs for the EIT problem. • We introduce a data-driven prior in the form of an EBM for improving the training convergence and accuracy of PINNs. • We propose a robust training framework for the EIT semi-inverse problem.

2. AN INVERSE PROBLEM WITH APPLICATION TO ELECTRICAL IMPEDANCE TOMOGRAPHY (EIT)

The inverse problem that we are interested in is inspired by applications in imaging paradigms such as electrical impedance tomography (EIT) and geophysical imaging for ground water flow.The EIT inverse problem reconstructs the unknown electrical conductivities σ of a body Ω ⊂ R d with d ∈ {2, 3} from measurements of finite electrical potential differences of neighboring surface electrodes. The differential equationgoverns the distribution of electric potential u in the body. Additionally, its accompanying BCs are given as following:An EIT experiment involves applying an electrical current on the surface ∂Ω of the region Ω to be imaged, which produces a current density σ ∂u ∂n | ∂Ω = g (Neumann data) where n is a unit normal vector w.r.t u associated with Ω at its boundary. The current also induces an electric potential u in the body, whose surface value u| ∂Ω = f can be measured. Thus by repeating several such experiments in which surface current is given and the corresponding surface voltage (Dirichlet data) is measured, we obtain the information on the Neumann-to-Dirichlet (NtD) operator which can be denoted by: Λ σ : g → f.(3)In the full EIT version, the problem is to reconstruct σ from just the surface current and voltage measurements. The estimate of the unknown conductivity σ can be reconstructed from a set of EIT experiments (Somersalo et al., 1992; Borcea, 2002; Hanke & Brühl, 2003) . In a simplified version that we attempt to solve here using our novel method, we will be interested in recovering σ from the measurements of u in the interior of the medium. This problem formulation is a close to the groundwater flow problem, wherein the source term on the right hand side of Eq. 1 is non-zero. It should also be noted, that a similar formulation was studied in Bar & Sochen (2021) .In our training paradigm, we need to simulate u for the interior of the medium for any underlying σ.We achieve that by first solving the forward problem by training neural network that can predict the value of the function u(x) at any given point x ∈ Ω, for any given σ. Once we have access to this pre-trained network that we will call u-Net, we will subsequently train another network to predict the value of σ(x) for any x ∈ Ω from point-wise measurements of a function u(x) that satisfies Eq. 1 and agrees with the Neumann and Dirichlet boundary data given from Eq. 3.

2.1. USING PINNS FOR EIT

Following the construction of physics-informed neural networks (PINNs), we can parameterize both σ and u using neural networks, called σ-Net and u-Net, respectively, and train them such that the values of σ-Net and u-Net for any provided point in the interior or boundary of the medium respect the PDE in Eq.1 or its boundary conditions 1 -depending on the position of the point. The forward

