EAGLE: LARGE-SCALE LEARNING OF TURBULENT FLUID DYNAMICS WITH MESH TRANSFORMERS

Abstract

Estimating fluid dynamics is classically done through the simulation and integration of numerical models solving the Navier-Stokes equations, which is computationally complex and time-consuming even on high-end hardware. This is a notoriously hard problem to solve, which has recently been addressed with machine learning, in particular graph neural networks (GNN) and variants trained and evaluated on datasets of static objects in static scenes with fixed geometry. We attempt to go beyond existing work in complexity and introduce a new model, method and benchmark. We propose EAGLE, a large-scale dataset of ∼1.1 million 2D meshes resulting from simulations of unsteady fluid dynamics caused by a moving flow source interacting with nonlinear scene structure, comprised of 600 different scenes of three different types. To perform future forecasting of pressure and velocity on the challenging EAGLE dataset, we introduce a new mesh transformer. It leverages node clustering, graph pooling and global attention to learn long-range dependencies between spatially distant data points without needing a large number of iterations, as existing GNN methods do. We show that our transformer outperforms state-of-the-art performance on, both, existing synthetic and real datasets and on EAGLE. Finally, we highlight that our approach learns to attend to airflow, integrating complex information in a single iteration.

1. INTRODUCTION

Despite consistently being at the center of attention of mathematics and computational physics, solving the Navier-Stokes equations governing fluid mechanics remains an open problem. In the absence of an analytical solution, fluid simulations are obtained by spatially and temporally discretizing differential equations, for instance with the finite volume or finite elements method. These simulations are computationally intensive, take up to several weeks for complex problems and require expert configurations of numerical solvers. Neural network-based physics simulators may represent a convenient substitute in many ways. Beyond the expected speed gain, their differentiability would allow for direct optimization of fluid mechanics problems (airplane profiles, turbulence resistance, etc.), opening the way to replace traditional trial-and-error approaches. They would also be an alternative for solving complex PDEs where numerical resolution is intractable. Yet, the development of such models is slowed down by the difficulty of collecting data in sufficient quantities to reach generalization. Velocity and pressure field measurements on real world systems require large and expensive devices, and simulation faces the problems described above. For all these reasons, few datasets are freely available for training highcapacity neural networks, and the existing ones either address relatively simple problems which can be simulated in reasonable time and exhibiting very similar behaviors (2D flow on a cylinder, airfoil Timestep t < l a t e x i t s h a 1 _ b a s e 6 4 = " m / + 2 W / V h 8 y F 7 6 w o n Z / T g f X K A X t 0 = " > A A A C 1 3 i c j V H L T s J A F D 3 U F + K r 4 t J N I z F x R Q q a 6 J L o x i U m v A w Q 0 p Y B J / S V d m o g h L g z b v 0 B t / p H x j / Q v / D O W B J d E L l N 2 z v n n n P m 3 h k 7 d H k s T P M j o 6 2 s r q 1 v Z D d z W 9 s 7 u 3 v 6 f r 4 R B 0 n k s L o T u E H U s q 2 Y u d x n d c G F y 1 p h x C z P d l n T H l 3 J e v O e R T E P / J q Y h K z r W U O f D 7 h j C Y J 6 e r 4 j 2 F h M a 9 x j s W C h M T N E T y + U i q Y K Y 3 F S Q B r V Q H 9 H B 3 0 E c J D A A 4 M P Q b k L C z E 9 b Z R g I i S s i y l h E W V c 1 R l m y J E 2 I R Y j h k X o i L 5 D W r V T 1 K e 1 9 I y V 2 q F d X H o j U h o 4 J k 1 A v I h y u Z u h 6 o l y l u g i 7 6 n y l L 1 N 6 G + n X h 6 h A n e E / q e b M 5 f V y V k E B r h Q M 3 C a K V S I n M 5 J X R J 1 K r J z 4 9 d U g h x C w m T e p 3 p E u a O U 8 3 M 2 l C Z W s 8 u z t V T 9 U z E l K t d O y k 3 w J b t c 7 o I b 5 W L p t F i + O S t U L t O r z u I Q R z i h + z x H B d e o o k 7 e Y 7 z g F W / a r f a g P W p P P 1 Q t k 2 o O 8 C e 0 5 2 + F S J b / < / l a t e x i t > Multi-Head Attention Timestep t + 1 < l a t e x i t s h a 1 _ b a s e 6 4 = " t v X v P F u M 0 m k S D + X q L f a 3 F r o w F D I = " > A A A C 2 X i c j V H L S s N A F D 2 N r 1 p f 8 b F z E y y C I J S 0 C r o s u n F Z o S 9 o S 0 n S a R 2 a F 8 l E r K U L d + L W H 3 C r P y T + g f 6 F d 8 Y U d F H s D U n u n H v O m X t n 7 N D l s T D N j 4 y 2 s L i 0 v J J d z a 2 t b 2 x u 6 d s 7 9 T h I I o f V n M A N o q Z t x c z l P q s J L l z W D C N m e b b L G v b w U t Y b t y y K e e B X x S h k H c 8 a + L z P H U s Q 1 N X 3 2 o L d i X G V e y w W L D Q m h j g u d v V 8 s W C q M G Y n e a R R C f R 3 t N F D A A c J P D D 4 E J S 7 s B D T 0 0 I R J k L C O h g T F l H G V Z 1 h g h x p E 2 I x Y l i E D u k 7 o F U r R X 1 a S 8 9 Y q R 3 a x a U 3 I q W B Q 9 I E x I s o l 7 s Z q p 4 o Z 4 n O 8 h 4 r T 9 n b i P 5 2 6 u U R K n B D 6 H + 6 K X N e n Z x F o I 9 z N Q O n m U K F y O m c 1 C V R p y I 7 N 3 5 N J c g h J E z m P a p H l D t K O T 1 n Q 2 l i N b s 8 W 0 v V P x V T o n L t p N w E X 7 L L + S 6 4 X i o U T w q l 6 9 N 8 + S K 9 6 i z 2 c Y A j u s 8 z l H G F C m r k f Y 8 X v O J N a 2 k P 2 q P 2 9 E P V M q l m F 3 9 C e / 4 G u j a X b w = = < / l a t e x i t > In this paper, we introduce EAGLE, a large-scale dataset for learning unsteady fluid mechanics. We accurately simulate the airflow produced by a two-dimensional unmanned aerial vehicle (UAV) moving in 2D environments with different boundary geometries. This choice has several benefits. It models the complex ground effect turbulence generated by the airflow of an UAV following a control law, and, up to our knowledge, is thus significantly more challenging than existing datasets. It leads to highly turbulent and non-periodic eddies, and high flow variety, as the different scene geometries generate completely different outcomes. At the same time, the restriction to a 2D scene (similar to existing datasets) makes the problem manageable and allows for large-scale amounts of simulations (∼1.1m meshes). The dataset will be made publically available upon publication. As a second contribution, we propose a new multi-scale attention-based model, which circumvents the quadratic complexity of multi-head attention by projecting the mesh onto a learned coarser representation yielding fewer but more expressive nodes. Conversely to standard approaches based on graph neural networks, we show that our model dynamically adapts to the airflow in the scene by focusing attention not only locally, but also over larger distances. More importantly, attention for specific heads seems to align with the predicted airflow, providing evidence of the capacity of the model to integrate long range dependencies in a single hop -see Figure 1 . We evaluate the method on several datasets and achieve state-of-the-art performance on two public fluid mechanics datasets (Cylinder-Flow, (Pfaff et al., 2021) and Scalar-Flow (Eckert et al., 2019) ), and on EAGLE.

2. RELATED WORK

Fluids datasets for deep learning -are challenging to produce in many ways. Real world measurement is complicated, requiring complex velocimetry devices (Wang et al., 2020; Discetti & Coletti, 2018; Erichson et al., 2020) . Remarkably, (Eckert et al., 2019; De Bézenac et al., 2019) leverage alignment with numerical simulation to extrapolate precise GT flows on real world phenomena (smoke clouds and sea surface temperature). Fortunately, accurate simulation data can by acquired through several solvers, ranging from computer graphics-oriented simulators (Takahashi et al., 2021; Pfaff & Thuerey, 2016) to accurate computational fluid dynamics solver (OpenFOAM © , Ansys © Fluent, ...). A large body of work (Chen et al., 2021a; Pfaff et al., 2021; Han et al., 2022; Stachenfeld et al., 2021; Pfaff et al., 2021) introduces synthetic datasets limited to simple tasks, such as 2D flow past a cylinder. EAGLE falls into this synthetic category, but differs in two main points: (a) simulations rely on hundreds of procedurally generated scene configurations, requiring several weeks of calculations on a high-performance computer, and (b) we used an engineer-grade fluid solver with demanding turbulence model and a fine domain discretization. For a comparison, see table 1. Learning of fluid dynamics -is mainly addressed with message passing networks. Recent work focuses in particular on smoothed-particle hydrodynamics (SPH) (Shao et al., 2022; Ummenhofer et al., 2020; Shlomi et al., 2021; Li et al., 2019; Allen et al., 2022) , somehow related to a Lagrangian representation of fluids. Sanchez-Gonzalez et al. (2020) proposes to chain graph neural networks in an Encode-Process-Decode pipeline to learn interactions between particles. SPH simulations are



Figure 1: We introduce EAGLE, a large-scale dataset for learning complex fluid mechanics, accurately simulating the air flow created by a 2D drone in motion and interacting with scenes of varying 2D geometries. We address the problem through an autoregressive model and self-attention over tokens in a coarser resolution, allowing to integrate long-range dependencies in a single hop -shown in the given example by the attention distributions for ◼, which follows the airflow.

