EDGE-VARYING FOURIER GRAPH NETWORKS FOR MULTIVARIATE TIME SERIES FORECASTING Anonymous authors Paper under double-blind review

Abstract

The key to multivariate time series (MTS) analysis and forecasting is to disclose the underlying couplings between variables that drive the co-movements. Considerable recent successful MTS methods are built with graph neural networks (GNNs) due to their essential capacity for relational modeling. However, previous work often used a static graph structure of time-series variables for modeling MTS, but failed to capture their ever-changing correlations over time. In this paper, we build a fully-connected supra-graph, representing non-static correlations between any two variables at any two timestamps, to capture high-resolution spatial-temporal dependencies. Whereas, conducting graph convolutions on such a supra-graph heavily increases computational complexity. As a result, we propose the novel Edge-Varying Fourier Graph Networks (EV-FGN), which reformulates the graph convolutions in the frequency domain with high efficiency and scale-free parameters, and applies edge varying graph filters to capture the time-varying variable dependencies. Extensive experiments show that EV-FGN outperforms state-of-the-art methods on seven real-world MTS datasets.



Recently, deep neural networks, such as long short-term memory (LSTM) Hochreiter & Schmidhuber (1997) , convolutional neural network (CNN) Borovykh et al. (2017 ), Transformer Vaswani et al. (2017) , have dominated MTS modeling. In particular, graph neural networks (GNNs) have demonstrated promising performance on MTS forecasting with their essential capability to capture the complex couplings between time-series variables. Some studies enable to adaptively learn the graph for MTS forecasting even without an explicit graph structure, e.g., by node similarity Mateos et al. ( 2019 Despite the success of GNNs on MTS forecasting, three practical challenges are eagerly demanded to address: 1) the dependencies between each pair of time series variables are generally non-static, which demands full dependencies modeling with all possible lags; 2) a high-efficiency dense graph learning method is demanded to replace the high-cost operators of graph convolutions and attention (with quadratic time complexity of the graph size); 3) the graph over MTS varies with different temporal interactions, which demands an efficient dynamic structure encoding method. In this paper, different from most GNN-based methods that construct graphs to model the spatial correlation between variables (i.e. nodes) Bai et al. (2020); Wu et al. (2019; 2020) , we attempt to build a supra-graph that sheds light on the "high-resolution" correlations between any two variables at any two timestamps (i.e., fine-grained spatial-temporal dependencies), and largely enhances the expressiveness on non-static spatial-temporal dependencies. Obviously, the supra-graph will heavily increase the computational complexity of GNN-based model, then a high-efficiency learning method is required to reduce the cost for model training. Inspired by Fourier Neural Operator (FNO) Li et al. ( 2021), we reformulate the graph convolution (time domain) to much lower-complexity matrix multiplication in the frequency domain by leveraging an efficient and newly-defined Fourier Graph Shift Operator (FGSO). In addition, it is necessary to consider the multiple iterations (layers) of graph convolutions to expand receptive neighbors and mix the diffusion information in the graph. To capture time-varying diffusion over the supra-graph, we introduce the edge-varying graph filter to weight the graph edges differently from different iterations and reformulate the edge-varying graph filter with multiple frequency-invariant FGSOs in the frequency domain to reduce the computational cost of graph convolutions. Finally, a novel Edge-Varying Fourier Graph Networks (EV-FGN) is designed for MTS analysis, which is stacked with multiple FGSOs to perform high-efficient multi-layer graph convolutions in the Fourier space. The main contributions of this paper are summarized as follows: • We adaptively learn a supra-graph, representing non-static correlations between any two variables at any two timestamps, to capture high-resolution spatial-temporal dependencies. • To efficiently compute graph convolutions over the supra-graph, we reformulate the graph convolutions in the Fourier space by leveraging FGSO. To the best of our knowledge, this work makes the first step to reformulate the graph convolutions in the Fourier space. • We design a novel network EV-FGN evolved from edge-varying graph filters for MTS analysis to capture the time-varying variable dependencies in the Fourier space. This study makes the first attempt to design a complex-valued feed-forward network in the Fourier space to efficiently compute multi-layer graph convolutions. • Extensive experimental results on seven MTS datasets demonstrate that EV-FGN achieves state-ofthe-art performance with high efficiency and fewer parameters. Multifaceted visualizations further interpret the efficacy of EV-FGN in graph representation learning for MTS forecasting. 2022) proposes an attention mechanism with low-rank approximation in frequency and a mixture of expert decomposition to control the distribution shift. However, these models only capture temporal dependencies in the frequency domain. StemGNN Cao et al. (2020) takes the advantages of both inter-series correlations and temporal dependencies by modeling them in the spectral domain, but, it captures the temporal and spatial dependencies separately. Unlike these efforts, our model is able to jointly encode spatial-temporal dependencies in the Fourier space.



(MTS) forecasting is a key ingredient in many real-world scenarios, including weather forecasting Zheng et al. (2015), decision making Borovykh et al. (2017), traffic forecasting Yu et al. (2018a); Bai et al. (2020), COVID-19 prediction Cao et al. (2020); Chen et al. (2022), etc.

); Bai et al. (2020); Wu et al. (2019) and/or self-attention mechanism Cao et al. (2020).

MULTIVARIATE TIME SERIES FORECASTING Classic time series forecasting methods are linear models, such as VAR Watson (1993), ARIMA Asteriou & Hall (2011) and state space model (SSM) Hyndman et al. (2008). Recently, deep learning based methods Lai et al. (2018); Sen et al. (2019); Zhou et al. (2021) have dominated MTS forecasting due to their capability of fitting any complex nonlinear correlations Lim & Zohren (2021). MTS with GNN. More recently, MTS have embraced GNN Wu et al. (2019); Bai et al. (2020); Wu et al. (2020); Yu et al. (2018b); Chen et al. (2022); Li et al. (2018) due to their best capability of modeling structural dependencies between variables. Most of these models, such as STGCN Yu et al. (2018b), DCRNN Li et al. (2018) and TAMP-S2GCNets Chen et al. (2022), require a pre-defined graph structure which is usually unknown in most cases. In recent years, some GNN-based works Kipf et al. (2018); Deng & Hooi (2021) account for the dynamic dependencies due to network design such as the time-varying attention Deng & Hooi (2021). In comparison, our proposed model captures the dynamic dependencies leveraging the high-resolution correlation in the supra-graph without introducing specific networks. MTS with Fourier transform. Recently, increasing MTS forecasting models have introduced the Fourier theory into neural networks as high-efficiency convolution operators Guibas et al. (2022); Chi et al. (2020). SFM Zhang et al. (2017) decomposes the hidden state of LSTM into multiple frequencies by discrete Fourier transform (DFT). mWDN Wang et al. (2018) decomposes the time series into multilevel sub-series by discrete wavelet decomposition and feeds them to LSTM network, respectively. ATFN Yang et al. (2022) utilizes a time-domain block to learn the trending feature of complicated non-stationary time series and a frequency-domain block to capture dynamic and complicated periodic patterns of time series data. FEDformer Zhou et al. (

