TILDE-Q: A TRANSFORMATION INVARIANT LOSS FUNCTION FOR TIME-SERIES FORECASTING

Abstract

Time-series forecasting has caught increasing attention in the AI research field due to its importance in solving real world problems across different domains, such as energy, weather, traffic, and economy. As shown in various types of data, it has been a must-see issue to deal with drastic changes, temporal patterns, and shapes in sequential data that previous models are weak in prediction. This is because most cases in time-series forecasting aims to minimize L p norm distances as loss functions, such as mean absolute error (MAE) or mean square error (MSE). These loss functions are vulnerable to not only consider temporal dynamics modeling, but also capture the shape of signals. In addition, these functions often make models misbehave and return uncorrelated results to the original time-series. To become an effective loss function, it has to be invariant to the set of distortions between two time-series data instead of just comparing exact values. In this paper, we propose a novel loss function, called TILDE-Q (Transformation Invariant Loss function with Distance EQuilibrium), that not only considers the distortions in amplitude and phase but also allows models to capture the shape of time-series sequences. In addition, TILDE-Q supports modeling periodic and non-periodic temporal dynamics at the same time. We evaluate the effectiveness of TILDE-Q by conducting extensive experiments with respect to periodic and non-periodic conditions of data, from naive models to state-of-the-art models. The experiment results indicate that the models trained with TILDE-Q outperforms those trained with other training metrics (e.g., MSE, dynamic time warping (DTW), temporal distortion index (TDI), and longest common subsequence (LCSS)).

1. INTRODUCTION

Time-series forecasting has been a core problem across various domains, including traffic domain (Li et al., 2018; Lee et al., 2020) , economy (Zhu & Shasha, 2002) , and disease propagation analysis (Matsubara et al., 2014) . The crucial part of the time-series forecasting is modeling of the complex temporal dynamics (e.g., non-stationary signal, periodicity). Temporal dynamics, intuitively, shape, has always been one of the most attention-getting keywords in time-series domains, such as rush hour of traffic data or abnormal usage of the electricity (Keogh et al., 2003; Bakshi & Stephanopoulos, 1994; Weigend & Gershenfeld, 1994; Wu et al., 2021; Zhou et al., 2022) A shape is a part of patterns in time-series data with a given time interval that could give valuable information, such as rise, drop, trough, peak, and plateau. We call the prediction is informative when it could properly consider the shape. In real-world applications like economics, such informative prediction is crucial to make decisions. To gain informative forecasting, the model should consider the shape rather than only aim to forecast accurate value for each time step. However, existing models do



. Deep learning methods are one of the appealing solutions to model complex non-linear temporal dependencies and nonstationary signals, but recent work reveals that even deep learning is often insufficient to model temporal dynamics. To properly model the temporal dynamics, Wu et al. (2021); Zhou et al. (2022) have proposed a novel deep learning approaches with input sequence decomposition. Le Guen & Thome (2019) try to model sudden changes timely and accurately with dynamic time warping (DTW). Bica et al. (2020) adopts domain adversarial training to learn balanced representations, which is a treatment invariant representations over time. Wu et al. (2021); Zhou et al. (2022) have less attention to the essence of the problem: a shape, in other words, temporal dynamics. Le Guen & Thome (2019); Bica et al. (2020) try to capture the shape but still have some limitations like Fig. 1 (c).

