IRREGULARITY REFLECTION NEURAL NETWORK FOR TIME SERIES FORECASTING

Abstract

Time series forecasting is a long-standing challenge in a variety of industries, and deep learning stands as the mainstream paradigm for handling this forecasting problem. With recent success, representations of time series components (e.g., trend and seasonality) are also considered in the learning process of the models. However, the residual remains under explored due to difficulty in formulating its inherent complexity. In this study, we propose a novel Irregularity Reflection Neural Network (IRN) that reflect the residual for the time series forecasting. First, we redefine the residual as the irregularity and express it as a sum of individual, short regular waves considering the Fourier series in a micro perspective. Second, we design a module, based on the convolutional architectures to mimic the variables of the derived irregularity representation, named Irregularity Representation Block (IRB). IRN comprises IRB on top of a forecasting model to learn the irregularity representation of time series. Extensive experiments on multiple realworld datasets demonstrate that IRN outperforms the state-of-the-art benchmarks in time series forecasting tasks.

1. INTRODUCTION

Figure 1 : The Traffic data and its time series components (i.e., trend, seasonality, and irregularity). Owing to the ubiquitous computing systems, time series is available in a wide range of domains including traffic (Chen et al., 2001) , power plant (Gensler et al., 2016) , stock market indices (Song et al., 2021) , and so on (Liu et al., 2015; Duan et al., 2021) . Spontaneously, interests in time series forecasting have grown, and as a result, an intensive research for a more accurate prediction. In recent literature, many deep learning models have been favored for forecasting problems (Lim & Zohren, 2021) . Recurrent Neural Network (RNN) and its extensions such as Long Short-Term Memory (LSTM) (Hochreiter & Schmidhuber, 1997) and Gated Recurrent Unit (GRU) (Chung et al., 2014) are popular choices for analyzing long sequences. Nevertheless, these models tend to be restricted in handling multivariate time series. As a powerful alternative, Convolution Neural Networks (CNNs) has been introduced to capture overall characteristics of time series through parallel calculations and filter operations. Building on the success in forecasting task, CNNbased models have been proposed according to the type of time series data. Temporal Convolutional Network (TCN) was applied to audio datasets (Oord et al., 2016) , whereas Graph Convolutional Network (GCN) was utilized in the time series with graph characteristics (e.g., human skeletonbased action recognition (Zhang et al., 2020) and traffic dataset (Bai et al., 2020) ). The attention models have also been applied to emphasize the specific sequence data that are primarily referenced when making the predictions (Liu et al., 2021b) . Despite the great efforts made, forecasting performance has room for further improvement as aforementioned models learn feature representations directly from complex real-world time series, often overlooking essential information. Recently, incorporating representations of time series components (e.g., trend, seasonality) used in conventional econometric approaches have shown to lead to better performances of the learning models. For instance, N-BEATS (Oreshkin et al., 2019) , Autoformer (Wu et al., 2021), and CoST (Woo et al., 2022) reflected the trend and seasonality of the time series and achieved improvements. However, as shown in Figure 1 , time series also include the irregularity that is not accounted by the trend and seasonality, and is yet under explored (Woo et al., 2022) . To address this challenge, we show how to deal with the irregularity of the time series data to improve the forecasting performance of the deep learning models. To this end, we represent the irregularity into an encodable expression on basis of Fourier series viewed from a micro perspective. The derived representation is encoded using convolutional architectures, and named as Irregularity Representation Block (IRB). Then, IRB embedded on a base model builds the Irregularity Reflection Neural Network (IRN). We demonstrate that IRN outperforms existing state-of-the-art forecasting models on eleven popular real-world datasets.

2.1. DEEP LEARNING FOR TIME SERIES FORECASTING

Sequential deep learning models such as RNN, LSTM, and GRU have long been used for time series forecasting (Elman, 1990; Hochreiter & Schmidhuber, 1997; Chung et al., 2014) . Although effective in capturing the temporal dependencies of time series, RNN-based models neglect the correlations in-between time series. To tackle this issue, Liu et al. ( 2020) propose a dual-stage two-phase (DSTP) to extract the spatial and temporal features. Shi et al. (2015) present convLSTM replacing the states of LSTM block with convolutional states. Another limitation of the sequential models are that the discrepancy between ground truth and prediction is accumulated over time as predictions are referred to predict further into the future (Liu et al., 2021a) . More recent works have demonstrated that CNNs can be applied in multivariate time series problems as well. Ravi et al. (2016) introduce the 1D convolution for human activity recognition, whereas Zhao et al. (2017) suggest the use of 2D convolution. CNN models are parallelizable, and hence show following advantages: the consideration of the correlation between variates and the prevention of error accumulation (Liu et al., 2019) . A downside is the limited receptive field when predicting long sequences due to the increasing number of the parameters (Zhao et al., 2017) . Wang et al. (2019) tackle this challenge by decomposing the long sequences according to long, short and closeness. CNN-based models have received increasing attention to enhance the forecasting performance. For example, the dilated casual convolutional layer is used to increase the receptive field by downsampling and improve long sequences prediction (Sen et al., 2019; Oord et al., 2016) . Another approach is Graph Convolutional Network (GCN), that analyzes the relation between nodes with specific position and edge relation, especially in traffic data (Fang et al., 2021; Song et al., 2020) and human body skeleton data (Yoon et al., 2022; Chen et al., 2021) . Attention-based models have also been adopted (Liu et al., 2019) and further developed into Transformer (Zhou et al., 2021; Liu et al., 2021b) . However, these approaches do not take into account the characteristics of time series such as trend, seasonality and irregularity.

2.2. REFLECTING THE REPRESENTATIVE COMPONENTS OF TIME SERIES

Considerable studies on time series analysis have relied on the decomposition of time series into non-random components. For instance, DeJong et al. (1992) conducted analysis on the trends of the macroeconomic time series as well as Lee & Shen (2009) emphasized the importance of obtaining significant trend relationship in linear time complexity. Jonsson & Eklundh (2002) extracted and analyzed the seasonality of the time series data and Taylor & Letham (2018) considered both trend and seasonality. When extracting these non-random components, a non-stationary time series becomes stationary, meaning time-independent. As conventional statistical methods such as ARIMA (Autoregressive Integrated Moving Average) (Williams & Hoel, 2003) and GP (Gaussian Process)

