LEARNING HETEROGENEOUS INTERACTION STRENGTHS BY TRAJECTORY PREDICTION WITH GRAPH NEURAL NETWORK

Abstract

Dynamical systems with interacting agents are universal in nature, commonly modeled by a graph of relationships between their constituents. Recently, various works have been presented to tackle the problem of inferring those relationships from the system trajectories via deep neural networks, but most of the studies assume binary or discrete types of interactions for simplicity. In the real world, the interaction kernels often involve continuous interaction strengths, which cannot be accurately approximated by discrete relations. In this work, we propose the relational attentive inference network (RAIN) to infer continuously weighted interaction graphs without any ground-truth interaction strengths. Our model employs a novel pairwise attention (PA) mechanism to refine the trajectory representations and a graph transformer to extract heterogeneous interaction weights for each pair of agents. We show that our RAIN model with the PA mechanism accurately infers continuous interaction strengths for simulated physical systems in an unsupervised manner. Further, RAIN with PA successfully predicts trajectories from motion capture data with an interpretable interaction graph, demonstrating the virtue of modeling unknown dynamics with continuous weights.

1. INTRODUCTION

Dynamical systems with interactions provide a fundamental model for a myriad of academic fields, yet finding out the form and strength of interactions remains an open problem due to its inherent degeneracy and complexity. Although it is crucial to identify the interaction graph of a complex system for understanding its dynamics, disentangling individual interactions from trajectory data without any ground-truth labels is a notoriously hard inverse problem. Further, if the interactions are heterogeneous and coupled with continuous strength constants, the interaction graph is called weighted and the inference became much harder with increased level of degeneracies. In this work, we assume the dynamical system with N objects (or agents), and their (discretized) trajectories x 1 , x 2 , . . . , x N from timestep t = 0 to T are given. If the system has an interaction kernel Q(x i , x j ) and the dynamics are governed by a form of ẋi = j =i k ij Q(x i , x j ) with some variable k ij , which is prevalent in nature and physical system, we call k ij as an interaction strength between the object i and j. With proper normalization, we can always regard 0 ≤ k ij ≤ 1. In general, k ij may have continuous values and forms a weighted interaction graph, which can be expressed in the form of a connectivity matrix K; a conventional adjacency matrix with continuous-valued entries of k ij . Hence, the problem is inferring continuous adjacency matrix K from trajectories x alone. In the current work, we propose a neural network called Relational Attentive Inference Network (RAIN) to address the problem of inferring weighted interaction graphs from multivariate trajectory data in an unsupervised manner. RAIN infers the interaction strength between two agents from previous trajectories by learning the attentive weight while simultaneously learning the unknown dynamics of the system and thus is able to precisely predict the future trajectories. Our model employs the attention mechanism twice: once for the construction of pairwise trajectory embedding

