ENHANCING THE INDUCTIVE BIASES OF GRAPH NEU-RAL ODE FOR MODELING PHYSICAL SYSTEMS

Abstract

Neural networks with physics-based inductive biases such as Lagrangian neural networks (LNN), and Hamiltonian neural networks (HNN) learn the dynamics of physical systems by encoding strong inductive biases. Alternatively, Neural ODEs with appropriate inductive biases have also been shown to give similar performances. However, these models, when applied to particle-based systems, are transductive in nature and hence, do not generalize to large system sizes. In this paper, we present a graph-based neural ODE, GNODE, to learn the time evolution of dynamical systems. Further, we carefully analyze the role of different inductive biases on the performance of GNODE. We show that similar to LNN and HNN, encoding the constraints explicitly can significantly improve the training efficiency and performance of GNODE significantly. Our experiments also assess the value of additional inductive biases, such as Newton's third law, on the final performance of the model. We demonstrate that inducing these biases can enhance the performance of the model by orders of magnitude in terms of both energy violation and rollout error. Interestingly, we observe that the GNODE trained with the most effective inductive biases, namely MCGNODE, outperforms the graph versions of LNN and HNN, namely, Lagrangian graph networks (LGN) and Hamiltonian graph networks (HGN) in terms of energy violation error by 4 orders of magnitude for a pendulum system, and 2 orders of magnitude for spring systems. These results suggest that NODE-based systems can give competitive performances with energy-conserving neural networks by employing appropriate inductive biases.

1. INTRODUCTION AND RELATED WORKS

Learning the dynamics of physical systems is a challenging problem that has relevance in several areas of science and engineering such as astronomy (motion of planetary systems), biology (movement of cells), physics (molecular dynamics), and engineering (mechanics, robotics) (LaValle, 2006; Goldstein, 2011) . The dynamics of a system are typically expressed as a differential equation, solutions of which may require the knowledge of abstract quantities such as force, energy, and drag (LaValle, 2006; Zhong et al., 2020; Sanchez-Gonzalez et al., 2020) . From an experimental perspective, the real observable for a physical system is its trajectory represented by the position and velocities of the constituent particles. Thus, learning the abstract quantities required to solve the equation, directly from the trajectory, can extremely simplify the problem of learning the dynamics (Finzi et al., 2020) . Infusing physical laws as prior has been shown to improve learning in terms of additional properties such as energy conservation, and symplectic nature (Karniadakis et al., 2021; Lutter et al., 2019; Liu et al., 2021) . To this extent, three broad approaches have been proposed, namely, Lagrangian neural networks (LNN) (Cranmer et al., 2020a; Finzi et al., 2020; Lutter et al., 2019) , Hamiltonian neural networks (HNN) (Sanchez-Gonzalez et al., 2019; Greydanus et al., 2019; Zhong et al., 2020; 2021) , and neural ODE (NODE) (Chen et al., 2018; Gruver et al., 2021) . The learning efficiency of LNNs and HNNs is shown to enhance significantly by employing explicit constraints (Finzi et al., 2020) and their inherent structure Zhong et al. (2019) . In addition, it has been shown that the superior performance of HNNs and LNNs is mainly due to their second-order bias, and not due to their symplectic or energy conserving bias (Gruver et al., 2021) . More specifically, an HNN with separable potential (V (q)) and kinetic (T (q, q)) energies is equivalent to a second order NODE of the form q = F (q, q). Thus, a NODE with a second-order bias can give similar performances to

