DEEPTIME: DEEP TIME-INDEX META-LEARNING FOR NON-STATIONARY TIME-SERIES FORECASTING

Abstract

Advances in I.T. infrastructure has led to the collection of longer sequences of time-series. Such sequences are typically non-stationary, exhibiting distribution shifts over time -a challenging scenario for the forecasting task, due to the problems of covariate shift, and conditional distribution shift. In this paper, we show that deep time-index models possess strong synergies with a meta-learning formulation of forecasting, displaying significant advantages over existing neural forecasting methods in tackling the problems arising from non-stationarity. These advantages include having a stronger smoothness prior, avoiding the problem of covariate shift, and having better sample efficiency. To this end, we propose Deep-Time, a deep time-index model trained via meta-learning. Extensive experiments on real-world datasets in the long sequence time-series forecasting setting demonstrate that our approach achieves competitive results with state-of-the-art methods, and is highly efficient. Code is attached as supplementary material, and will be publicly released.

1. INTRODUCTION

Time-series forecasting has important applications across business and scientific domains, such as demand forecasting (Carbonneau et al., 2008) , capacity planning and management (Kim, 2003) , and anomaly detection (Laptev et al., 2017) . With the advances of I.T. infrastructure, time-series are collected over longer durations, and at a higher sampling frequency. This has led to time-series spanning tens-of-thousands to millions of time steps, on which we would like to perform forecasting on. Such datasets face the unique challenge of non-stationarity, where long sequences face distribution shifts over time, due to factors like concept drift. This has practical implications on forecasting models, which face a degradation in performance at test time (Kim et al., 2021) due to covariate shift, and conditional distribution shift (see Appendix B for formal definitions). Table 1 : Time-index models are defined to be models whose predictions, ŷt , are purely functions of the current time-index features, τ t , e.g. relative time-index (1, 2, 3, ...), datetime features (minute-of-hour, week-of-day, etc.). Historical-value models, whose predictions of future time step(s), ŷt+1 , are explicit functions of past observations, (y t , y t-1 , . . .), and optionally covariates, (z t+1 , z t , z t-1 , . . .), which can include exogenous time-series or even datetime features.

Time-index Models

Historical-value Models ŷt = f (τ t ) ŷt+1 = f (y t , y t-1 , . . . , z t+1 , z t , . . .) E.g.: DeepTime, Prophet, Gaussian process E.g.: N-HiTS, Autoformer, DeepAR In this work, we posit that deep time-index models exhibit strong synergies with a meta-learning formulation to tackle the problem of non-stationary forecasting, whereas existing neural forecasting methods, which are historical-value models, are unable to take full advantage of this formulation, and are still susceptible to the problem of covariate shift. In the following, we discuss time-index models and their deep counterparts, highlighting how simple deep time-index models are unable to perform forecasting (i.e. extrapolate from historical training data). Yet, endowing them with a meta-learning formulation solves this problem. Thereafter, we demonstrate the advantages of deep time-index meta-learning for non-stationary forecasting and how they alleviate the issues faced by historical-value models, which are namely: (i) meta-learning is an effective solution for conditional distribution shift, (ii) they avoid the problem of covariate shift, (iii) have stronger sample efficiency, and (iv) that time-index models have a stronger smoothness prior.

