LEARNING CONTINUOUS-TIME PDES FROM SPARSE DATA WITH GRAPH NEURAL NETWORKS

Abstract

The behavior of many dynamical systems follow complex, yet still unknown partial differential equations (PDEs). While several machine learning methods have been proposed to learn PDEs directly from data, previous methods are limited to discretetime approximations or make the limiting assumption of the observations arriving at regular grids. We propose a general continuous-time differential model for dynamical systems whose governing equations are parameterized by message passing graph neural networks. The model admits arbitrary space and time discretizations, which removes constraints on the locations of observation points and time intervals between the observations. The model is trained with continuous-time adjoint method enabling efficient neural PDE inference. We demonstrate the model's ability to work with unstructured grids, arbitrary time steps, and noisy observations. We compare our method with existing approaches on several well-known physical systems that involve first and higher-order PDEs with state-of-the-art predictive performance.

1. INTRODUCTION

We consider continuous dynamical systems with a state u(x, t) ∈ R that evolves over time t ∈ R + and spatial locations x ∈ Ω ⊂ R D of a bounded domain Ω. We assume the system is governed by an unknown partial differential equation (PDE) u(x, t) := du(x, t) dt = F (x, u, ∇ x u, ∇ 2 x u, . . .), where the temporal evolution u of the system depends on the current state u and its spatial first and higher-order partial derivatives w.r.t. the coordinates x. Such PDE models are the cornerstone of natural sciences, and are widely applicable to modelling of propagative systems, such as behavior of sound waves, fluid dynamics, heat dissipation, weather patterns, disease progression or cellular kinetics (Courant & Hilbert, 2008) . Our objective is to learn the differential F from data. There is a long history of manually deriving mechanistic PDE equations for specific systems (Cajori, 1928) , such as the Navier-Stokes fluid dynamics or the Schrödinger's quantum equations, and approximating their solution forward in time numerically (Ames, 2014). These efforts are complemented by data-driven approaches to infer any unknown or latent coefficients in the otherwise known equations (Isakov, 2006; Berg & Nyström, 2017; Santo et al., 2019) , or in partially known equations (Freund et al., 2019; Seo & Liu, 2019b; Seo et al., 2020) . A series of methods have studied neural proxies of known PDEs for solution acceleration (Lagaris et al., 1998; Raissi et al., 2017; Weinan & Yu, 2018; Sirignano & Spiliopoulos, 2018) et al., 2019) . These models are fundamentally limited to discretizing the input domain with a sample-inefficient grid, while they also do not support continuous evolution over time, rendering them unable to handle temporally or spatially sparse or non-uniform observations commonly encountered in realistic applications. Models such as (Battaglia et al., 2016; Chang et al., 2016; Sanchez-Gonzalez et al., 2018) are related to the interaction networks where object's state evolves as a function of its neighboring objects, which forms dynamic relational graphs instead of grids. In contrast to the dense solution fields of PDEs, these models apply message-passing between small number of moving and interacting objects, which deviates from PDEs that are strictly differential functions. In 

2. METHODS

In this Section we consider the problem of learning the unknown function F from observations (y(t 0 ), . . . , y(t M )) ∈ R N ×(M +1) of the system's state u(t) = (u(x 1 , t), . . . , u(x N , t)) T at N arbitrary spatial locations (x 1 , . . . , x N ) and at M + 1 time points (t 0 , . . . , t M ). We introduce efficient graph convolution neural networks surrogates operating over continuous-time to learn PDEs from sparse data. Note that while we consider arbitrarily sampled spatial locations and time points, we do not consider the case of partially observed vectors y(t i ) i.e. when data at some location is missing at some time point. Partially observed vectors, however, could be accounted by masking the nodes with missing observations when calculating the loss. The function F is assumed to not depend on global values of the spatial coordinates i.e. we assume the system does not contain position-dependent fields (Section 2.1). We apply the method of lines (MOL) (Schiesser, 2012) to numerically solve Equation 1. The MOL consists of selecting N nodes in Ω and discretizing spatial derivatives in F at these nodes. We



Poli et al. (2019)  graph neural ordinary differential equations (GNODE) were proposed as a framework for modeling continuous-time signals on graphs. The main limitations of this framework in application to learning PDEs are the lack of spatial information about physical node locations and lack of motivation for why this type of model could be suitable. Our work can be viewed as connecting graph-based continuous-time models with data-driven learning of PDEs in spatial domain through a classical PDE solution technique.Contributions.In this paper we propose to learn free-form, continuous-time, a priori fully unknown PDE model F from sparse data measured on arbitrary timepoints and locations of the coordinate domain Ω with graph neural networks (GNN). Our contributions are:• We introduce continuous-time representation and learning of the dynamics of PDE-driven systems • We propose efficient graph representation of the domain structure using the method of lines with message passing neural networks • We achieve state-of-the-art learning performance on realistic PDE systems with irregular data, and our model is highly robust to data sparsity Scripts and data for reproducing the experiments can be found in this github repository. Comparison of machine-learning based PDE learning methods.

