NEURAL FIELD DISCOVERY DISENTANGLES EQUIV-ARIANCE IN INTERACTING DYNAMICAL SYSTEMS

Abstract

Systems of interacting objects often evolve under the influence of underlying field effects that govern their dynamics, e.g. electromagnetic fields in physics, or map topologies and traffic rules in traffic scenes. While the interactions between objects depend on local information, the underlying fields depend on global states. Pedestrians and vehicles in traffic scenes, for example, follow different traffic rules and social norms depending on their absolute geolocation. The entanglement of global and local effects makes recently popularized equivariant networks inapplicable, since they fail to capture global information. To address this, in this work, we propose to disentangle local object interactions -which are equivariant to global roto-translations and depend on relative positions and orientations-from external global field effects -which depend on absolute positions and orientations. We theorize the presence of latent fields, which we aim to discover without directly observing them, but infer them instead from the dynamics alone. We propose neural fields to learn the latent fields, and model the interactions with equivariant graph networks operating in local coordinate frames. We combine the two components in a graph network that transforms field effects in local frames and operates solely there. Our experiments show that we can accurately discover the underlying fields in charged particles settings, traffic scenes, and gravitational n-body problems, and effectively use them to learn the system and forecast future trajectories.

Input Trajectories

Discovered Field Figure 1 : N-body system simulation with underlying gravitational field. We uncover fields that underlie interacting systems using trajectories only. Systems of interacting objects are omnipresent in nature, with examples ranging from the subatomic to the astronomical scale -including colliding particles and n-body systems of celestial objects-as well as human-centric settings like traffic scenes, governed by social dynamics. The majority of these systems does not evolve in a vacuum, they instead evolve under the influences of underlying fields. For example, electromagnetic fields may govern the dynamics of charged particles. In traffic scenes, the road network and traffic rules govern the actions of traffic scene participants. N-body systems might swirl around supermassive black holes that create gravitational fields. Earlier work on learning interacting systems proposed graph networks (Kipf et al., 2018; Battaglia et al., 2016; Sanchez-Gonzalez et al., 2020) , while state-of-the-art methods for interacting systems propose equivariant graph networks (Walters et al., 2021; Satorras et al., 2021; Kofinas et al., 2021; Brandstetter et al., 2022) to model dynamics while respecting their underlying symmetries. These networks exhibit increased robustness and performance, while maintaining parameter efficiency due to weight sharing. They are, however, not compatible with underlying field effects, since they can only capture local states, such as relative positions, while fields depend on absolute states (positions or orientations). In other words, global fields violate the strict equivariance hypothesis. We are, thus, in need of an augmented notion of equivariance that encapsulates both local and global effects. In many real-world settings, strict SE(3) equivariance, -equivariance to the special Euclidean group of translations and rotations-does not hold. A function f that predicts trajectories in interacting systems

