SPATIOTEMPORAL MODELING OF MULTIVARIATE SIG-NALS WITH GRAPH NEURAL NETWORKS AND STRUC-TURED STATE SPACE MODELS

Abstract

Multivariate signals are prevalent in various domains, such as healthcare, transportation systems, and space sciences. Modeling spatiotemporal dependencies in multivariate signals is challenging due to (1) long-range temporal dependencies and (2) complex spatial correlations between sensors. To address these challenges, we propose representing multivariate signals as graphs and introduce GRAPHS4MER, a general graph neural network (GNN) architecture that captures both spatial and temporal dependencies in multivariate signals. Specifically, (1) we leverage Structured State Spaces model (S4), a state-of-the-art sequence model, to capture long-term temporal dependencies and (2) we propose a graph structure learning layer in GRAPHS4MER to learn dynamically evolving graph structures in the data. We evaluate our proposed model on three distinct tasks and show that GRAPHS4MER consistently improves over existing models, including (1) seizure detection from electroencephalography signals, outperforming a previous GNN with self-supervised pretraining by 3.1 points in AUROC; (2) sleep staging from polysomnography signals, a 4.1 points improvement in macro-F1 score compared to existing sleep staging models; and (3) traffic forecasting, reducing MAE by 8.8% compared to existing GNNs and by 1.4% compared to Transformer-based models.

1. INTRODUCTION

Multivariate signals are time series data measured by multiple sensors and are prevalent in many realworld applications, including healthcare (Mincholé et al., 2019) , transportation systems (Ermagun & Levinson, 2018 ), power systems (Negnevitsky et al., 2009) , and space sciences (Camporeale et al., 2018) . An example multivariate signal is scalp electroencephalograms (EEGs), which measure brain electrical activities using sensors placed on an individual's scalp. Several challenges exist in modeling spatiotemporal dependencies in multivariate signals. First, many types of signals are sampled at a high sampling rate, which results in long sequences that can be up to tens of thousands of time steps. Moreover, multivariate signals often involve long-range temporal correlations (Berthouze et al., 2010) . Prior studies on modeling long signals often preprocess the raw signals using frequency transformations (Tang et al., 2022b; Asif et al., 2020; Shoeibi et al., 2021; Covert et al., 2019; Guillot et al., 2020; Guillot & Thorey, 2021) or divide the signals into short windows and aggregate model predictions post-hoc (Phan & Mikkelsen, 2022; Pradhan et al., 2022) . However, such preprocessing steps may discard important information encoded in raw signals, as well as neglect long-range temporal dependencies in the signals. Therefore, a model that is capable of modeling long-range temporal correlations in raw signals is needed. Deep sequence models, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), and Transformers, have specialized variants for handling long sequences (Arjovsky et al., 2016; Erichson et al., 2021; Katharopoulos et al., 2020; Choromanski et al., 2021) . However, they struggle to scale to long sequences of tens of thousands of time steps (Tay et al., 2020) . Recently, the Structured State Space sequence model (S4) (Gu et al., 2022) , a deep sequence model based on the classic state space model, has achieved state-of-the-art performance on challenging long

