TILDE-Q: A TRANSFORMATION INVARIANT LOSS FUNCTION FOR TIME-SERIES FORECASTING

Abstract

Time-series forecasting has caught increasing attention in the AI research field due to its importance in solving real world problems across different domains, such as energy, weather, traffic, and economy. As shown in various types of data, it has been a must-see issue to deal with drastic changes, temporal patterns, and shapes in sequential data that previous models are weak in prediction. This is because most cases in time-series forecasting aims to minimize L p norm distances as loss functions, such as mean absolute error (MAE) or mean square error (MSE). These loss functions are vulnerable to not only consider temporal dynamics modeling, but also capture the shape of signals. In addition, these functions often make models misbehave and return uncorrelated results to the original time-series. To become an effective loss function, it has to be invariant to the set of distortions between two time-series data instead of just comparing exact values. In this paper, we propose a novel loss function, called TILDE-Q (Transformation Invariant Loss function with Distance EQuilibrium), that not only considers the distortions in amplitude and phase but also allows models to capture the shape of time-series sequences. In addition, TILDE-Q supports modeling periodic and non-periodic temporal dynamics at the same time. We evaluate the effectiveness of TILDE-Q by conducting extensive experiments with respect to periodic and non-periodic conditions of data, from naive models to state-of-the-art models. The experiment results indicate that the models trained with TILDE-Q outperforms those trained with other training metrics (e.g., MSE, dynamic time warping (DTW), temporal distortion index (TDI), and longest common subsequence (LCSS)).

1. INTRODUCTION

Time-series forecasting has been a core problem across various domains, including traffic domain (Li et al., 2018; Lee et al., 2020) , economy (Zhu & Shasha, 2002) , and disease propagation analysis (Matsubara et al., 2014) . The crucial part of the time-series forecasting is modeling of the complex temporal dynamics (e.g., non-stationary signal, periodicity). Temporal dynamics, intuitively, shape, has always been one of the most attention-getting keywords in time-series domains, such as rush hour of traffic data or abnormal usage of the electricity (Keogh et al., 2003; Bakshi & Stephanopoulos, 1994; Weigend & Gershenfeld, 1994; Wu et al., 2021; Zhou et al., 2022) A shape is a part of patterns in time-series data with a given time interval that could give valuable information, such as rise, drop, trough, peak, and plateau. We call the prediction is informative when it could properly consider the shape. In real-world applications like economics, such informative prediction is crucial to make decisions. To gain informative forecasting, the model should consider the shape rather than only aim to forecast accurate value for each time step. However, existing models do In this work, we aim to design a novel objective function that guides models to improve forecasting performance by learning the shapes in time-series data. To design such shape-aware loss function, we review existing literature (Esling & Agon, 2012; Bakshi & Stephanopoulos, 1994; Keogh, 2003) and investigate the notions of shapes and distortions that interrupt measurement for recognizing similarity of two time-series data in terms of shapes (Sec. 3.1, Sec. 3.2, and Sec. 3.3) . Based on the investigation, we newly propose required conditions for constructing an objective function for shape-aware time-series forecasting (Sec. 3.4). We then present a novel loss function, TILDE-Q (Transformation Invariant Loss function with Distance EQualibrium), that enables shape-aware representation learning with three different loss terms, which are invariant to the distortions (Sec. 4). For evaluation, we conduct extensive experiments with state-of-the-art deep learning models for time-series forecasting with TILDE-Q. The results indicate that TILDE-Q is model-agnostic and could improve accuracy of existing models, compared to MSE and DILATE. Contributions We make the following contributions: (1) To understand shape-awareness and distortion invariances in time-series forecasting, we investigate existing distortions in amplitude and phase; (2) we implement TILDE-Q that has invariances to many existing distortions and achieves shape-awareness and informative forecasting in a timely manner; and (3) we show that the proposed TILDE-Q allows models to have higher accuracy compared to those with existing metrics such as DTW, TDI, and LCSS on average.

2.1. TIME-SERIES FORECASTING

There are many methods for time-series forecasting from traditional ones, such as ARIMA model (Box et al., 2015) and hidden markov model (Pesaran et al., 2004) to recent deep learning models. In this section, we briefly describe the recent deep learning models for time-series forecasting. Starting with the huge success of the recurrent neural networks (RNNs) (Clevert et al., 2016; Li et al., 2018; Yu et al., 2017) , researchers have developed novel deep learning architectures, improving forecasting performance. To effectively capture long-term dependency, which is a weakness of RNNs, Stoller et al. (2020) have proposed convolutional neural networks (CNNs). However, it is required



. Deep learning methods are one of the appealing solutions to model complex non-linear temporal dependencies and nonstationary signals, but recent work reveals that even deep learning is often insufficient to model temporal dynamics. To properly model the temporal dynamics, Wu et al. (2021); Zhou et al. (2022) have proposed a novel deep learning approaches with input sequence decomposition. Le Guen & Thome (2019) try to model sudden changes timely and accurately with dynamic time warping (DTW). Bica et al. (2020) adopts domain adversarial training to learn balanced representations, which is a treatment invariant representations over time. Wu et al. (2021); Zhou et al. (2022) have less attention to the essence of the problem: a shape, in other words, temporal dynamics. Le Guen & Thome (2019); Bica et al. (2020) try to capture the shape but still have some limitations like Fig. 1 (c).

Figure 1: Ground-truth and forecasting results of Informer model with three training metrics (top) TILDE-Q, (middle) MSE, and (bottom) DTW-based loss function. (middle) MSE tends to generate non-informative forecasting results, similar to an average value of data and (bottom) DTW often produces misaligned results. Red dotted box contains three training metrics.

