METAPHYSICA: CAUSALITY-AWARE ROBUSTNESS TO OOD INITIAL CONDITIONS IN PHYSICS-INFORMED MACHINE LEARNING

Abstract

A fundamental challenge in physics-informed machine learning (PIML) is the design of robust PIML methods for out-of-distribution (OOD) forecasting tasks, where the tasks require learning-to-learn from observations of the same (ODE) dynamical system with different unknown parameters, and demand accurate forecasts even under initial conditions outside the training support. In this work we propose a solution for such tasks, which we define as a meta-learning procedure for causal structural discovery (including invariant risk minimization). Using three different OOD tasks, we empirically observe that the proposed approach significantly outperforms existing state-of-the-art PIML and deep learning methods.

1. INTRODUCTION

Physics-informed machine learning (PIML) (e.g., (Willard et al., 2020; Xingjian et al., 2015; Lusch et al., 2018; Yeo & Melnyk, 2019; Raissi et al., 2018; Kochkov et al., 2021) ) seeks to combine the strengths of physics and machine learning models and has positively impacted fields as diverse as biological sciences (Yazdani et al., 2020) , climate science (Faghmous & Kumar, 2014) , turbulence modeling (Ling et al., 2016; Wang et al., 2020a) , among others. PIML achieves substantial success in tasks where the test data comes from the same distribution as the training data (in-distribution tasks). Unlike the PIML work described above, this paper considers an out-of-distribution (OOD) change in the initial system state of the dynamical system, possibly with different train and test distribution supports (illustrated in Figure 1(a, b )). In this setting, we observe that existing state-of-the-art PIML models perform significantly worse than their performance in-distribution, even in PIML methods designed with OOD robustness in mind (Wang et al., 2021b; Kirchmeyer et al., 2022) . This is because the standard ML part of PIML, which tends to learn spurious associations, will perform poorly in our OOD setting. We then propose a promising solution: Combine meta learning with causal structure discovery to learn an ODE model that is robust to OOD initial conditions. In our OOD tasks, OOD robustness means that the robustness is tied to interventions over the initial conditions of the system, not on arbitrary interventions as the system evolves from the initial state. This is an important distinction. There can be multiple ODE models that will be equally OOD robust, and robust ODEs may not correctly predict system trajectories under arbitrary system interventions besides the initial state (Rubenstein et al. (2016) discusses the effect of arbitrary interventions in physics models). Contributions This work proposes a hybrid transductive-inductive modeling approach learning for more robust ODEs using meta learning and causal structure discovery (e.g., via L 1 regularization (Zheng et al., 2018) , which can be combined with invariant risk minimization (Arjovsky et al., 2019; Krueger et al., 2021) ). More precisely, our contributions are: 1. We 



show that state-of-the-art PIML and deep learning methods fail in test examples with OOD initial conditions. Prior work(Wang et al., 2021a)  showed that deep learning-only methods fail in OOD tasks, and argued physics models and PIML methods would succeed, including a proposed OOD solution(Wang et al., 2021b). Here we show that PIML methods also fail (or perform poorly) OOD, including the solution inWang et al. (2021b).

