PHASE2VEC: DYNAMICAL SYSTEMS EMBEDDING WITH A PHYSICS-INFORMED CONVOLUTIONAL NETWORK

Abstract

Dynamical systems are found in innumerable forms across the physical and biological sciences, yet all these systems fall naturally into equivalence classes: conservative or dissipative, stable or unstable, compressible or incompressible. Predicting these classes from data remains an essential open challenge in computational physics on which existing time-series classification methods struggle. Here, we propose, phase2vec, an embedding method that learns highquality, physically-meaningful representations of low-dimensional dynamical systems without supervision. Our embeddings are produced by a convolutional backbone that extracts geometric features from flow data and minimizes a physicallyinformed vector field reconstruction loss. The trained architecture can not only predict the equations of unseen data, but also produces embeddings that encode meaningful physical properties of input data (e.g. stability of fixed points, conservation of energy, and the incompressibility of flows) more faithfully than standard blackbox classifiers and state-of-the-art time series classification techniques. We additionally apply our embeddings to the analysis of meteorological data, showing we can detect climatically meaningful features. Collectively, our results demonstrate the viability of embedding approaches for the discovery of dynamical features in physical systems.

1. INTRODUCTION

The application of deep neural networks to the prediction (Lusch et al., 2018) , control (Haluszczynski & Räth, 2021) , and basic understanding (Raissi et al., 2019) of dynamical systems has spurred important advances across the sciences, from neuroscience (Sussillo et al., 2016) to physics (Karniadakis et al., 2021) , from earth and climate science (Reichstein et al., 2019; Fresca et al., 2020) to computational biology (Sapoval et al., 2022) . However, while deep learning has already contributed significantly to the analysis of single systems, its role in understanding the underlying principles of dynamical systems in general remains somewhat limited. The ability to predict or control one system, after all, tells us little about how to do so with another. Crucially, without the ability to generalize across behaviors of numerous dynamical systems, the problem of constructing new systems is quite difficult, and most current attempts rely on exhaustive search through system parameters (Hart et al., 2012; Scholes et al., 2019) . ) layer before being mapped to a set of estimated coefficients, Ξ recon . These coefficients are used to weight a dictionary of polynomials in q variables (depicted q = 2), and the resulting linear combination (depicted transposed above) comprises the estimated governing equation, F recon (X, Ξ recon ) associated to the reconstructed vector field. A machine learning framework that could elucidate the latent dynamical structure across numerous systems having different governing equations and parameters would be an important first step in this direction. However, progress on this front has been limited, largely because representations of even a single dynamical system are necessarily high-dimensional (many initial conditions over time) and the problem of understanding single systems in detail is already as daunting as it is useful. The few existing approaches that analyze multiple systems simultaneously focus on the limited setting of rapid adaptation to new parameter regimes for governing equations (Kirchmeyer et al., 2022) or rely on hardwired features and laborious search over model manifolds (Quinn et al., 2021) . To address this challenge, we propose phase2vec, a dynamical systems embedding model which learns high-quality, low-dimensional, physically-meaningful representations of dynamical systems (Fig. 1 ). We focus here on two-and three-dimensional dynamics because of their preponderance in nature, ranging from classical non-linear models of population growth like the Lotka-Volterra model to increasingly important models of climate dynamics (for applications to both, see Sec. 4). Similarly to many word embedding approaches (Mikolov et al., 2013) , which approximate the meaning of words using statistical regularity instead of formal semantics, our dynamical embeddings seek to model the "semantics" of dynamical systems from data instead of via analytical investigation. phase2vec is so-called since it is based on a vector of convolutional features extracted from the vector field representing the dynamics in phase space of input data. To encourage physicallymeaningful solutions from our encodings, we use a decoder which reconstructs input dynamics from a library of basis functions. The full pipeline is trained with a reconstruction loss placing special emphasis on fixed points, giving phase2vec a physically-informed "learning bias" (following the terminology for principles of physics-informed learning suggested in (Karniadakis et al., 2021), Box 2). Like other embedding methods, we train our system on an auxiliary task, in our case one involving equation reconstruction. We show that this auxiliary task, combined with a physics-informed loss, steers the network towards learning dynamically stable, informative embeddingsfoot_0 . The main contributions of this paper are: 1. phase2vec, a novel neural-network-based embedding method that can be used to learn low-dimensional representations of dynamical systems in an unsupervised manner. 2. A demonstration that these embeddings can be used to decode the governing equations of testing data, recover sparse underlying models, and denoise input data in a way that pre-



Code available here: https://github.com/nitzanlab/phase2vec



Figure1: phase2vec pipeline. phase2vec learns high-quality embeddings of dynamical systems from phase space data. An input vector field (represented as a stream plot, i.e. a set of trajectories) is passed through a series of convolutional, rectification, and downsampling layers. Terminal convolutional features are aggregated in a relatively low-dimensional (d = 100) layer before being mapped to a set of estimated coefficients, Ξ recon . These coefficients are used to weight a dictionary of polynomials in q variables (depicted q = 2), and the resulting linear combination (depicted transposed above) comprises the estimated governing equation, F recon (X, Ξ recon ) associated to the reconstructed vector field.

