NOVEL FEATURE REPRESENTATION STRATEGIES FOR TIME SERIES FORECASTING WITH PREDICTED FU-TURE COVARIATES

Abstract

Accurate time series forecasting is a fundamental challenge in data science. Unlike traditional statistical methods, conventional machine learning models, such as RNNs and CNNs, use historical data consisting of previously measured variables including the forecast variable and all its covariates. However, in many applications, some of the covariates can be predicted with reasonable accuracy for the immediate future. We refer to such covariates as predictable future covariates. Note that the input may also contain some covariates that cannot be accurately predicted. We consider the problem of predicting water levels at a given location in a river or canal system using historical data and future covariates, some of which (precipitation, tide) may be predictable. In many applications, for some covariates of interest, it may be possible to use historical data or accurate predictions for the near future. Traditional methods to incorporate future predictable covariates have major limitations. The strategy of simply concatenating the future predicted covariates to the input vector is highly likely to miss the past-future connection. Another strategy that iteratively predicts one step at a time can end up with prediction error accumulation. We propose two novel feature representation strategies to solve those limitations -shifting and padding, which create a framework for contextually linking the past with the predicted future, while avoiding any accumulation of prediction errors. Extensive experiments on three well-known datasets revealed that our strategies when applied to RNN and CNN backbones, outperform existing methods. Our experiments also suggest a relationship between the amount of shifting and padding and the periodicity of the time series.

1. INTRODUCTION

Conventional time series forecasting is widely used to predict a set of target variables at a future time point based on past data collected over a predetermined length. Next-step forecasting (Montgomery et al., 2015; Shi et al., 2022) refers to predicting the target variables at a time point one step into the future where the unit of time is the time granularity of the measurements. Multi-horizon forecasting (Quaedvlieg, 2021) predicts the target variables multiple steps into the future Capistrán et al. (2010) . Accurate forecasting allows people to do better resource management and optimization decisions for critical processes (Cinar et al., 2017; Salinas et al., 2020; Rangapuram et al., 2018) . Applications include probabilistic demand forecasting in retail (Böse et al., 2017) , dynamic assignments of beds to patients (Zhang & Nawata, 2018), monthly inflation forecasting, and much more. Good multi-horizon forecasting requires historical data of the target variables from which to learn long-term patterns. In addition, it also requires measurements from heterogeneous data sources of useful covariates, often from the recent past. However, in many applications, some of the covariates can also be predicted with reasonable accuracy for the immediate future. We refer to such covariates as future covariates. For example, in some applications, a covariate of interest could be "precipitation", for which it is possible to use historical data as well as reasonably accurate predictions for the near future, which may be obtained from the weather service. Despite its importance, only limited approaches exist that use future covariates to improve time series predictions. Related methods can be mainly categorized into direct strategy using sequence-to-sequence models (Mariet & Kuznetsov, 2019) and iterated methods using autoregressive models (Sahoo et al., 2020) .

