Φ-DVAE: LEARNING PHYSICALLY INTERPRETABLE REPRESENTATIONS WITH NONLINEAR FILTERING

Abstract

Incorporating unstructured data into physical models is a challenging problem that is emerging in data assimilation. Traditional approaches focus on well-defined observation operators whose functional forms are typically assumed to be known. This prevents these methods from achieving a consistent model-data synthesis in configurations where the mapping from data-space to model-space is unknown. To address these shortcomings, in this paper we develop a physics-informed dynamical variational autoencoder (Φ-DVAE) for embedding diverse data streams into timeevolving physical systems described by differential equations. Our approach combines a standard (possibly nonlinear) filter for the latent state-space model and a VAE, to embed the unstructured data stream into the latent dynamical system. A variational Bayesian framework is used for the joint estimation of the embedding, latent states, and unknown system parameters. To demonstrate the method, we look at three examples: video datasets generated by the advection and Korteweg-de Vries partial differential equations, and a velocity field generated by the Lorenz-63 system. Comparisons with relevant baselines show that the Φ-DVAE provides a data efficient dynamics encoding methodology that is competitive with standard approaches, with the added benefit of incorporating a physically interpretable latent space.

1. INTRODUCTION

Physical models -as represented by ordinary, stochastic, or partial differential equations -are ubiquitous throughout engineering and the physical sciences. These differential equations are the synthesis of scientific knowledge into mathematical form. However, as a description of reality they are imperfect (Judd & Smith, 2004) , leading to the well-known problem of model misspecification (Box, 1979) . At least since Kalman (1960) physical modellersls with observations (Anderson & Moore, 1979) . Such approaches are usually either solving the inverse problem of attempting to recover model parameters from data, and/or, the data assimilation (DA) problem of conducting state inference based on a time-evolving process. For the inverse problem, Bayesian methods are common (Tarantola, 2005; Stuart, 2010) . In this, prior belief in model parameters Λ is updated with data y to give a posterior distribution, p(Λ|y). This describes uncertainty with parameters given the data and modelling assumptions. DA can also proceed from a Bayesian viewpoint, where inference is cast as a nonlinear state-space model (SSM) (Law et al., 2015; Reich & Cotter, 2015) . The SSM is typically the combination of a time-discretised differential equation and an observation process: uncertainty enters the model through extrusive, additive errors. For a latent state variable u n representing some discretised system at time n, with observations y n , the object of interest is the filtering distribution p(u n |y 1:n ), where y 1:n := {y k } n k=1 . Additionally, the joint filtering and estimation problem, which estimates p(u n , Λ|y 1:n ) has received significant attention in the literature (see, e.g., Kantas et al. (2015) and references therein). This has been well studied in, e.g., electrical engineering (Storvik, 2002) , geophysics (Bocquet & Sakov, 2013) , neuroscience (Ditlevsen & Samson, 2014 ), chemical engineering (Kravaris et al., 2013 ), biochemistry (Dochain, 2003 ), and hydrology (Moradkhani et al., 2005) , to name a few. Typically in data assimilation tasks, while parameters of an observation model may be unknown, the observation model itself is assumed known (Kantas et al., 2015) . This assumption breaks down in settings where data arrives in various modalities, such as videos, images, or audio, hindering the

