GRAPH SPLINE NETWORKS FOR EFFICIENT CONTINU-OUS SIMULATION OF DYNAMICAL SYSTEMS

Abstract

While complex simulations of physical systems have been widely studied in engineering and scientific computing, lowering their often prohibitive computational requirements has only recently been tackled by deep learning approaches. In this paper, we present GRAPHSPLINENETS, a novel deep learning approach to speed up simulation of physical systems with spatio-temporal continuous outputs by exploiting the synergy between graph neural networks (GNN) and orthogonal spline collocation (OSC). Two differentiable OSC (time-oriented OSC and spatial-oriented OSC) are applied to bridge the gap between discrete GNN outputs and generate continuous solutions at any location in space and time without explicit prior knowledge of underlying differential equations. Moreover, we introduce an adaptive collocation strategy in space to enable the model to sample from the most important regions. Our model improves on widely used graph neural networks for physics simulation on both efficiency and solution accuracy. We demonstrate GRAPHSPLINENETS in predicting complex dynamical systems such as the heat equation, damped wave propagation and the Navier-Stokes equations for incompressible flows, where they improve accuracy of more than 25% while providing at least 60% speedup.

1. INTRODUCTION

For a growing variety of fields, simulations of partial differential equations (PDEs) representing physical processes are an essential tool. PDE-based simulators have been widely employed in a range of practical issues, spanning from astrophysics (Mücke et al., 2000) to biology (Quarteroni & Veneziani, 2003) , engineering (Wu & Porté-Agel, 2011 ), finance, (Marriott et al., 2015) or weather forecasting (Bauer et al., 2015) . Traditional solvers for phsysics-based simulation oftentimes need a significant amount of computational resources (Houska et al., 2012) , such as solvers based on first principles and the modified Gauss-Newton methods. To broaden the scope of applications of dynamics simulation, the scientific machine learning community has put considerable effort into developing computationally simple yet accurate simulation approaches. Deep learning has been shown to be a powerful alternative to efficiently compute solutions (Raissi et al., 2019) or model dynamical systems directly from data (Mrowca et al., 2018) . Among deep learning methods, graph neural networks (GNNs) come with desirable properties such as spatial equivariance and translational invariance which allow learning representations of dynamical interactions in a generalizable manner (Pfaff et al., 2021; Bronstein et al., 2021) and on unstructured grids. Despite the benefits of these paradigms, graph-based models have the fundamental drawback of being discrete in nature, which makes it challenging to implement continuous simulations in time and space. While graph models that operate in continuous space or continuous time have been introduced in the past (Poli et al., 2019) , such approaches mainly deal with only one aspect of continuity at once, either in space or time, and are hindered by accuracy issues while interpolating in space (Alet et al., 2019) or require a considerable number of iterative evaluations of a vector field in time limiting their performance (Xhonneux et al., 2020) . To bridge the gap between the inherently discrete graphs and the intrinsic continuous nature of the real world, in this work we propose GRAPHSPLINENETS, a novel method that exploits the synergy between graph neural networks and the orthogonal spline collocation (OSC) method (Bialecki & Fairweather, 2001; Fairweather & Meade, 2020) . By leveraging the OSC, our approach can produce predictions at any location in space and time without explicit prior knowledge of the underlying differential equation. GRAPHSPLINENETS achieve significant speedups compared to GNN baselines by making use of efficient sparse linear solvers (de Boor & Weiss, 1980) for the OSC problem and training the model end-to-end with larger temporal resolutions. Moreover, thanks to the super-convergent approximations at nodes of the OSC partition (Qiao et al., 2021) and an adaptive sampling strategy of collocation points, GRAPHSPLINENETS improve the solution accuracy of predictions in continuous space and time. We summarize our contributions as follows: • We introduce GRAPHSPLINENETS, a learning framework for complex dynamical system in continuous time and space leveraging the OSC method. • We introduce an adaptive collocation sampling strategy to improve accuracy and a differentiable algorithm for fast inference of the OSC that allows for end-to-end training. • We demonstrate that GRAPHSPLINENETS outperform or are competitive against baselines in predicting continuous complex dynamics in terms of both accuracy and speed.

2. RELATED WORKS

Graph Neural Networks for Dynamics Predictions Deep neural networks have recently been successfully employed in a variety of different tasks, ranging from simulated (Long et al., 2018; Li et al., 2020) and real datasets (Pathak et al., 2022; Li et al., 2022a; Poli et al., 2022) demonstrating their capabilities in predicting complex dynamics often orders of magnitude faster than traditional numerical solvers. We aim at finding efficient and accurate surrogate models: unlike data-driven approaches for solving PDEs such as PINNs (Raissi et al., 2019) , that aim at finding solutions to a set of equations, our methods does not need to know the exact equations of a dynamical system and can directly learn mappings from data. One major line of work for dynamics prediction involves the use of graph neural networks (GNNs): these models provide several benefits compared to other deep learning methods based on regular grids such as convolutional networks. In particular, they make it possible to learn on irregular grids and varying connectivity and inherit physical properties derived from geometric deep learning, such as permutation and spatial equivariance (Bronstein et al., 2021) . GNN-based approaches have also been shown to represent some parts of classical numerical solvers, such as finite differences and volumes (Brandstetter et al., 2022; Lienen & Günnemann, 2022) . Graph neural networks have also recently been applied to large-scale weather predictions (Keisler, 2022) . Another deep learning direction on irregular grids that avoids graphs altogether is to convert the input domain into a regular grid via learnable deformations to make usage of neural operators possible (Li et al., 2022b) . Compared to other deep learning methods for physics predictions on unstructured grids, we do not need to learn transforms and predict directly in the target domains; moreover, we place emphasis on bridging the gap between discrete graph nodes in space and time by allowing for fast and accurate continuous predictions. Collocation Methods and Graphs Collocation and interpolation methodsfoot_0 are used to estimate unknown data values from known ones (Bourke, 1999) . GNNs for predicting dynamics inherently lack an important aspect characterizing physical systems: continuity. The concept of continuity can be separated into two categories: continuity in time and continuity in space. The former has been investigated by using continuous ODE models (Poli et al., 2019; Xhonneux et al., 2020; Fang et al., 2021) that can, in theory, represent a system evolving in time continuously. However, such methods employ numerical solvers that introduce a considerable number of function evaluations and do not consider the fact that deep learning models can be capable of overcoming time discretization



Collocation and interpolation are terms that are oftentimes used interchangeably. While interpolation is defined as obtaining unknown values from known ones, collocation is usually defined as a finite solution space satisfying equations dictated by known (collocation) points. Thus, collocation can be considered as a flexible subset of interpolation methods that satisfies certain conditions, such as C 1 class continuity.



Alet et al. (2019) represent adaptively sampled points in a graph architecture to simulate continuous underlying physical processes without any a priori graph structure. Sanchez-Gonzalez et al. (2020) introduce particle-based graph simulators with dynamically changing connectivity simulating interactions through message passing; Pfaff et al. (2021) extend particle-based simulations to mesh-based ones.

