CRITICAL SAMPLING FOR ROBUST EVOLUTION BE-HAVIOR LEARNING OF UNKNOWN DYNAMICAL SYS-TEMS

Abstract

We study the following new and important problem: given an unknown dynamical system, what is the minimum number of samples needed for effective learning of its governing laws and accurate prediction of its future evolution behavior, and how to select these critical samples? In this work, we propose to explore this problem based on a design approach. Specifically, starting from a small initial set of samples, we adaptively discover and collect critical samples to achieve increasingly accurate learning of the system evolution. One central challenge here is that we do not know the network modeling error of the ground-truth system state, which is however needed for critical sampling. To address this challenge, we introduce a multi-step reciprocal prediction network where a forward evolution network and a backward evolution network are designed to learn and predict the temporal evolution behavior in the forward and backward time directions, respectively. Very interestingly, we find that the desired network modeling error is highly correlated with the multi-step reciprocal prediction error. More importantly, this multi-step reciprocal prediction error can be directly computed from the current system state without knowing the ground-truth or data statistics. This allows us to perform a dynamic selection of critical samples from regions with high network modeling errors and develop an adaptive sampling-learning method for dynamical systems. To achieve accurate and robust learning from this small set of critical samples, we introduce a joint spatial-temporal evolution network which incorporates spatial dynamics modeling into the temporal evolution prediction for robust learning of the system evolution operator with few samples. Our extensive experimental results demonstrate that our proposed method is able to dramatically reduce the number of samples needed for effective learning and accurate prediction of evolution behaviors of unknown dynamical systems by up to hundreds of times, especially for high-dimensional dynamical systems.

1. INTRODUCTION

Recently, learning-based methods for complex and dynamic system modeling have become an important area of research in machine learning. The behaviors of dynamical systems in the physical world are governed by their underlying physical laws (Bongard & Lipson, 2007; Schmidt & Lipson, 2009) . In many areas of science and engineering, ordinary differential equations (ODEs) and partial differential equations (PDEs) play important roles in describing and modeling these physical laws (Brunton et al., 2016; Raissi, 2018; Long et al., 2018; Chen et al., 2018; Raissi et al., 2019; Qin et al., 2019) . In recent years, data-driven modeling of unknown physical systems from measurement data has emerged as an important area of research. There are two major approaches that have been explored. The first approach typically tries to identify all the potential terms in the unknown governing equations from a priori dictionary, which includes all possible terms that may appear in the equations (Brunton et al., 2016; Schaeffer & McCalla, 2017; Rudy et al., 2017; Raissi, 2018; Long et al., 2018; Wu & Xiu, 2019; Wu et al., 2020; Xu & Zhang, 2021) . The second approach for data-driven learning of unknown dynamical systems is to approximate the evolution operator of the underlying equations, instead of identifying the terms in the equations (Qin et al., 2019; Wu & Xiu, 2020; Qin et al., 2021a; Li et al., 2021b) .

