REPRESENTING MULTI-VIEW TIME-SERIES GRAPH STRUCUTRES FOR MULTIVARIATE LONG-TERM TIME-SERIES FORECASTING

Abstract

Multivariate long-term time-series forecasting task is a very challenging task in real-world application areas, such as electricity consumption and influenza-like illness forecasting. At present, researchers are focusing on designing robust and effective models, and have achieved good results. However, there are several issues with existing models that need to be overcome to ensure they provide optimal performance. First, the lack of a relationship structure between multivariate variables needs to be addressed. Second, most models only have a weak ability to capture local dynamic changes across the entire long-term time-series. And, third, the current models suffer from high computational complexity and unsatisfactory accuracy. To address these issues, we propose a novel method called Multi-view Time-series Graph Structure Representation (MTGSR) for multivariate long-term time-series forecasting tasks. MTGSR uses graph convolutional networks (GCNs) to construct topological relationships in the multivariate long-term time-series from three different perspectives: time, dimension, and crossing segments. Variation trends in the different dimensions of the multivariate long-term time-series are extracted through a difference operation so as to construct a topological map that reflects the correlations between the different dimensions. Then, to capture the dynamically changing characteristics of the fluctuation correlations between adjacent local sequences, MTGSR constructs a cross graph by calculating the correlation coefficients between adjacent local sequences. Extensive experiments on five different datasets show that MTGSR reduces errors by 20.41% over the state-of-the-art while maintaining linear complexity. Additionally, memory use is decreased by 66.52% and running time is reduced by 78.09%.

1. INTRODUCTION

In reality, a large amount of time-series data is produced in various fields, such as weather forecasting (Hewage et al., 2021; Rasp et al., 2020) , electricity power planning (Qader et al., 2022; Oreshkin et al., 2021) , disease propagation prejudgment (Li et al., 2021; Zimmer & Yaesoubi, 2020) , and more. Although challenging to model the long-term relationships and multivariate correlations within these real-world time-series are important elements of most practical forecasting tasks involving these data. Thus, in this paper, we focus on multivariate long-term time-series forecasting task, which has higher requirements for models than ordinary time-series forecasting tasks. In recent years, deep learning models have been thoroughly investigated for their power at multivariate longseries forecasting tasks with many achieving good results (Liu et al., 2021; Torres et al., 2021; Lim & Zohren, 2021) . For example, Transformer-based models, the mainstream framework for multivariate long-term time-series forecasting tasks, relies on multi-head self-attention as a core mechanism for extracting powerful characteristics from historical data (Nikita et al., 2020; Zhou et al., 2021; Xu et al., 2021; Zhou et al., 2022) . These characteristics are then analyzed to predict long sequences containing data from farther in the future. However, there are still several extremely challenging issues in multivariate long-term time-series forecasting tasks that need to be addressed. First, existing models do not construct relationships between multivariate variables. Rather, they pay more attention to capturing the temporal features of the series, which means they simply use dimensional mappings to extract blurry relationships between multivariate variables. Topologies between different variables cannot effectively be constructed using this approach. Second, in addition to the relationship between tokens, the characteristics of dynamically changing fluctuations between local sequences in long-term time-series are also important. Yet most existing models process sequences from a global view such that the features of the local fluctuations are entangled with the overall features. Third, the current models still have room to improve accuracy. And, further, most of the models that perform well have a high computational complexity caused by complex structures. To construct a relationship graph of multivariate variables, we turned to graph convolutional networks (GCNs) (Welling & Kipf, 2016) . GCNs are typical graph neural networks used to extract the features of vertices connected by edges in a graph. One advantage of GCNs is that they can generate a more representative topology and richer node properties by passing information between neighboring nodes. Hence, we attempted to build a model by treating the multivariate variables in a multivariate long-term time-series as the nodes of a graph and using the correlations between different variables fluctuating over time as the weights of the graph's edges. Through experiments, we found that these operations could generate dimensional graphs with rich spatial features. Moreover, the GCNs could be used to extract more appropriate topological features between multivariate variables from the obtained dimensional graphs. Additionally, we subsequently found that the process of generating graphs from the time-series and the GCNs was also good for extracting several other graph characteristics-including the temporal characteristics of the long-term time-series and the characteristics of the local sub-sequences with dynamically changing fluctuations. Inspired by these preliminary studies, we developed a novel and effective model named Multi-view time-series Graph Structure Representation (MTGSR) for multivariate long-term time-series forecasting tasks. MTGSR extracts disentangled information from the input time-series to dynamically generate graph structures from three perspectives: the time-view, the dimension-view, and a crossview. In terms of the time-view, MTGSR builds a time graph using Time Graph Generator by calculating the correlations for all the dimensional information between different timestamps. Unlike the normal process of generating a time graph, MTGSR adds a differential operation to process the inputs when generating dimensional graphs so as to extract valid information from the relative fluctuations of the time-series across different dimensions. The cross-view takes into account the correlations of all the fluctuations that dynamically change over time between two adjacent local sequences in the long-term time-series. Because extracting features directly from the entire length of the time-series would result in too many redundant information and probably cause the model to overfit, the objects MTGSR's three graph generators use are local sequences split from the intact multivariate long-term time-series. Benefiting from this design, the scale of the parameters is greatly reduced to the point that the model has a linear complexity. Further, Inspired by Transformer's multihead attention mechanism, a multi-head mechanism is used at MTGSR's input stage to improve its ability to capture different features from the input sequence. This strategy proves to increase the prediction accuracy of the model. In fact, MTGSR outperforms the state-of-the-art model on five data benchmarks in terms of accuracy, memory use, and running times. The contributions of this paper are summarized as follows: • We propose a novel model named Multi-view Time-series Graph Structure Representation (MTGSR) for multivariate long-term time-series forecasting tasks. MTGSR uses GCNs to learn the complex disentangled characteristics in multivariate long-term time-series from three perspectives: the time view, the dimension view, and the cross-segment view. • To construct topologies between multivariate variables, MTGSR uses a GCN-based Dimension Graph Generator to dynamically learn the structural relationships in the multivariate long-term time-series after differencing operations. • To capture the dynamically changing characteristics of the fluctuation correlations between adjacent local sequences in the whole long-term time-series, MTGSR construct a crosssegment graph by calculating the correlation coefficients between adjacent local sequences through the Cross-segments Graph Generator. • Extensive experiments with five datasets show that MTGSR reduces errors by 20.41% while maintaining a linear complexity compared to the state-of-the-art framework FEDformer. Additionally, MTGSR reduces memory use by 66.52% and running time by 78.09%.

