BI-STRIDE MULTI-SCALE GRAPH NEURAL NETWORK FOR MESH-BASED PHYSICAL SIMULATION

Abstract

Learning physical systems on unstructured meshes by flat Graph neural networks (GNNs) faces the challenge of modeling the long-range interactions due to the scaling complexity w.r.t. the number of nodes, limiting the generalization under mesh refinement. On regular grids, the convolutional neural networks (CNNs) with a U-net structure can resolve this challenge by efficient stride, pooling, and upsampling operations. Nonetheless, these tools are much less developed for graph neural networks (GNNs), especially when GNNs are employed for learning large-scale mesh-based physics. The challenges arise from the highly irregular meshes and the lack of effective ways to construct the multi-level structure without losing connectivity. Inspired by the bipartite graph determination algorithm, we introduce Bi-Stride Multi-Scale Graph Neural Network (BSMS-GNN) by proposing bi-stride as a simple pooling strategy for building the multi-level GNN. Bi-stride pools nodes by striding every other Breadth-First-Search (BFS) frontier; it 1) works robustly on any challenging mesh in the wild, 2) avoids using a mesh generator at coarser levels, 3) avoids the spatial proximity for building coarser levels, and 4) uses non-parametrized aggregating/returning instead of MLPs during pooling and unpooling. Experiments show that our framework significantly outperforms the state-of-the-art method's computational efficiency in representative physics-based simulation cases.

1. INTRODUCTION

Simulating physical systems through numerically solving partial differential equations (PDEs) plays a key role in various science and engineering applications, ranging from particle-based (Jiang et al., 2016) and mesh-based (Li et al., 2020a) solid mechanics to grid-based fluid (Bridson, 2015) and aero (Cao et al., 2022) dynamics. Despite the extensive successes in improving their stability, accuracy, and efficiency, numerical solvers are often computationally expensive for time-sensitive applications, especially iterative design optimization where fast online inferring is desired. Recently, machine learning approaches have demonstrated impressive potential in improving the efficiency of inferring physical states with competitive accuracy. Representative methods include end-to-end frameworks (Obiols-Sales et al., 2020) and those with physics-informed neural networks (PINNs) (Raissi et al., 2019; Karniadakis et al., 2021; Sun et al., 2020; Gao et al., 2021) . Many existing works apply convolutional neural networks (CNNs) (Fukushima & Miyake, 1982) to learn physical systems on two-or three-dimensional structured grids (Kim et al., 2019; Fotiadis et al., 2020; Gao et al., 2021; Guo et al., 2016; Tompson et al., 2017) . It is generally recognized that CNNs exhibit strong performance on handling local information with convolution and global information with pooling/upsampling. However, the strict dependency on regular domain shapes makes it non-trivial to be applied on unstructured meshes. Although it is possible to deform the domains to rectangular shapes to apply CNNs (Gao et al., 2021) or other models, such as NeuralOpera-torNets (Li et al., 2022) , the challenge remains for domains with complex topologies, which are common in practice. On the other hand, graph neural networks (GNNs) have been considered as a natural choice for physics-based simulation on unstructured meshes (Battaglia et al., 2018; Belbute-Peres et al., 2020; Gao et al., 2022; Harsch & Riedelbauch, 2021; Pfaff et al., 2020; Sanchez-Gonzalez et al., 2018; 2020) . However, all the above methods use the flat GNN that faces two challenges when the graph size increases: (1) Oversmoothing: the graph convolution can be seen as a low-pass filter that suppresses the signal with higher frequency than a certain value (Chen et al., 2020; Li et al., 2020b) . Multiple passes of graph convolution then become an iterative projection onto the eigenspace of the graph where all higher frequency signals are smoothed out, which also makes training harder. (2) Complexity: Under mesh refinement, not only that more nodes are there to be processed, but the message passing (MP) iterations also grow linearly to propagate information to the same physical distance (Fortunato et al., 2022) . As a result, a quadratic complexity becomes inevitable for both the running time and the memory to store the computational graph. To mitigate these limitations, researchers recently start investigating multi-scale GNNs (MS-GNNs) for physics-based simulation (Fortunato et al., 2022; Li et al., 2020b; Lino et al., 2021; Liu et al., 2021; Lino et al., 2022a; b) . The multi-scale approach is appealing as it tackles the oversmoothing issue by building sub-level graphs on coarser resolutions, which lead to longer range interaction and naturally fewer MP times. However, pooling and adjacency building should be conducted carefully to avoid introducing partitions into the coarser levels (Gao & Ji, 2019), which stops information exchange across the separated clusters. Existing solutions include utilizing the spatial proximity for building the connections at the coarser levels (Lino et al., 2021; Liu et al., 2021; Lino et al., 2022a; b) , or generating coarser meshes for the original geometry (Fortunato et al., 2022; Liu et al., 2021) , and randomly pooling nodes then applying Nyström approximation for the original adjacency matrix (Li et al., 2020b) . However, all of them suffer from limitations: the spatial proximity can result in wrong connections across the geometry boundaries; the mesh generation is laboring and often unavailable for unseen meshes; and the random pooling may introduce partitions in the coarser levels. We observe that all the aforementioned limitations originate from pooling and building connections at coarser levels. To the best of our knowledge, no existing work can systematically generate multiscale GNNs with arbitrary levels for an arbitrary geometry in the wild while completely avoiding cutting or wrong connections across the boundaries. To this end, in this work, we introduce a simple yet robust and effective pooling strategy, bi-stride. Bi-stride is inspired by the bi-partition determination in DAG (directed acyclic graph). It pools all nodes on every other BFS (breadth-first-search) frontiers, such that a 2 nd -powered adjacency enhancement conserves all the connectivity. We also accompany bi-stride with a non-parameterized aggregating/returning method to handle the transition between adjacent levels to decrease the model complexity. Our framework, namely Bi-Stride Multi-Scale Graph Neural Network (BSMS-GNN), is tested on three benchmarks (CYLINDERFLOW, AIRFOIL, and DEFORMINGPLATE) from GraphMeshNets and INFLATINGFONT, a new dataset of inflating elastic surfaces with many self-contacts. In all cases, BSMS-GNN shows a dominant advantage in memory footprint and required training and inference time compared to alternatives.

2. BACKGROUND AND RELATED WORKS

GNNs for Physics-Based Simulation GNNs are first applied to physical simulation to learn the behaviors of particle systems, deformable solids, and Lagrangian fluids (Battaglia et al., 2016; Chang et al., 2016; Mrowca et al., 2018; Sanchez-Gonzalez et al., 2020) . Notably, the generalized Message Passing (Sanchez-Gonzalez et al., 2018 ) is broadly accepted for information propagation. Based on that, GraphMeshNets (Pfaff et al., 2020) sets a milestone for learning mesh-based simulation. Following GraphMeshNets, which predicts a single forward timestep, there have been several variants, including 1) solving forward and inverse problems by combining GNNs with PINNs (Gao et al., 2022) , 2) predicting long-term system states combined with GraphAutoEncoder (GAE) and Transformer (Han et al., 2022) , 3) predicting steady states with multi-layer readouts (Harsch & Riedelbauch, 2021) , and 4) up-sampling from coarser meshes with differentiable simulation (Belbute-Peres et al., 2020 ). Yet still, with flat GNNs, the quadratic computation complexity on finer meshes poses great challenges. We claim that adopting a multi-level structure is an effective solution. Multi-Scale GNNs It is common to apply GNNs with multi-level structures in various graphrelated tasks, such as graph classification (Wu et al., 2020; Mesquita et al., 2020; Zhang et al., 2019) . GraphUNet (GUN) (Gao & Ji, 2019) first introduces the UNet structures into GNN with a trainable scoring module for pooling; it also has a 2 nd -powered adjacency enhancement to reduce the chance of losing connectivity. A few works have investigated multi-scale GNNs (MS-GNNs) for physics-based simulation. Specifically, Fortunato et al. (2022) and Liu et al. (2021) define twoand multi-level GNNs, respectively, for physics-based simulation, but both of them rely on pre-

