LEARNING CONTINUOUS-TIME PDES FROM SPARSE DATA WITH GRAPH NEURAL NETWORKS

Abstract

The behavior of many dynamical systems follow complex, yet still unknown partial differential equations (PDEs). While several machine learning methods have been proposed to learn PDEs directly from data, previous methods are limited to discretetime approximations or make the limiting assumption of the observations arriving at regular grids. We propose a general continuous-time differential model for dynamical systems whose governing equations are parameterized by message passing graph neural networks. The model admits arbitrary space and time discretizations, which removes constraints on the locations of observation points and time intervals between the observations. The model is trained with continuous-time adjoint method enabling efficient neural PDE inference. We demonstrate the model's ability to work with unstructured grids, arbitrary time steps, and noisy observations. We compare our method with existing approaches on several well-known physical systems that involve first and higher-order PDEs with state-of-the-art predictive performance.

1. INTRODUCTION

We consider continuous dynamical systems with a state u(x, t) ∈ R that evolves over time t ∈ R + and spatial locations x ∈ Ω ⊂ R D of a bounded domain Ω. We assume the system is governed by an unknown partial differential equation (PDE) u(x, t) := du(x, t) dt = F (x, u, ∇ x u, ∇ 2 x u, . . .), where the temporal evolution u of the system depends on the current state u and its spatial first and higher-order partial derivatives w.r.t. the coordinates x. Such PDE models are the cornerstone of natural sciences, and are widely applicable to modelling of propagative systems, such as behavior of sound waves, fluid dynamics, heat dissipation, weather patterns, disease progression or cellular kinetics (Courant & Hilbert, 2008) . Our objective is to learn the differential F from data. There is a long history of manually deriving mechanistic PDE equations for specific systems (Cajori, 1928) , such as the Navier-Stokes fluid dynamics or the Schrödinger's quantum equations, and approximating their solution forward in time numerically (Ames, 2014). These efforts are complemented by data-driven approaches to infer any unknown or latent coefficients in the otherwise known equations (Isakov, 2006; Berg & Nyström, 2017; Santo et al., 2019) , or in partially known equations (Freund et al., 2019; Seo & Liu, 2019b; Seo et al., 2020) . A series of methods have studied neural proxies of known PDEs for solution acceleration (Lagaris et al., 1998; Raissi et al., 2017; Weinan & Yu, 2018; Sirignano & Spiliopoulos, 2018) or for uncertainty quantification (Khoo et al., 2017) .  Related



work. Recently the pioneering work ofLong et al. (2017)  proposed a fully non-mechanistic method PDE-Net, where the governing equation F is learned from system snapshot observations as a convolutional neural network (CNN) over the input domain discretised into a spatio-temporal grid. Further works have extended the approach with residual CNNs (Ruthotto & Haber, 2019), symbolic neural networks(Long et al., 2019), high-order autoregressive networks (Geneva & Zabaras, 2020),

