COPULA CONFORMAL PREDICTION FOR MULTI-STEP TIME SERIES FORECASTING

Abstract

Accurate uncertainty measurement is a key step to building robust and reliable machine learning systems. Conformal prediction is a distribution-free uncertainty quantification algorithm popular for its ease of implementation, statistical coverage guarantees, and versatility for underlying forecasters. However, existing conformal prediction algorithms for time series are limited to single-step prediction without considering the temporal dependency. In this paper we propose a Copula Conformal Prediction algorithm for multivariate, multi-step Time Series forecasting, CopulaCPTS. On several synthetic and real-world multivariate time series datasets, we show that CopulaCPTS produces more calibrated and sharp confidence intervals for multi-step prediction tasks than existing techniques.

1. INTRODUCTION

Deep learning models are becoming widely used in high-risk settings such as healthcare, transportation, and finance. In these settings, it is important that a model produces calibrated uncertainty to reflect its own confidence and to assist decision making. Confidence regions are a common approach to quantify forecast uncertainty (Khosravi et al., 2011) . A (1 -α)-confidence region Γ 1-α for a random variable y is valid if it contains y's true value with high probability: P(y ∈ Γ 1-α ) ≥ 1 -α. Note one can make Γ 1-α infinitely wide to satisfy validity. For the confidence region to be useful, we want to minimize its area while remaining valid; this is known as the efficiency of the region. Conformal prediction is a powerful method that produces confidence regions with finite-sample guarantees of validity (Vovk et al., 2005; Lei et al., 2018) . Furthermore, it makes no assumptions about the forecast model or the underlying data distribution. Its generality, simplicity, and statistical guarantees have made conformal prediction popular for many real world applications including drug discovery (Eklund et al., 2015) and robotics (Luo et al., 2021) . In this paper we present a more calibrated and efficient conformal prediction algorithm for multi-step times series forecasting. This type of problem is ubiquitous -examples include predicting hurricane paths, vehicle trajectories, and financial or epidemic forecasts. To quantify uncertainty, we want a "cone of uncertainty" that covers the entirety of the forecast horizon. Stankevičiūtė et al. (2021) presented an algorithm, CF-RNN, for multi-step time series with the assumption that all time steps are modeled independently. In practice, however, we found that CF-RNN produces confidence regions often too large to be useful, especially when the problem is multivariate or has a long forecast horizon. We introduce CopulaCPTS, a Copula-based Conformal Prediction algorithm for multi-step Time Series forecasting. We improve efficiency by utilizing copulas to model the dependency between forecasted time steps. A copula is a multivariate cumulative distribution function that models the dependence between multiple random variables. It is widely used in economic forecasting (Nelsen, 2007; Patton, 2012) and has been introduced to conformal algorithms by Messoudi et al. (2021) to model correlation between multiple targets in non-temporal settings. Copula processes have been explored in generative models for time series (Salinas et al., 2019; Drouin et al., 2022) . We found that copulas are effective in capturing uncertainty relations between time steps. Our contributions are: • CopulaCPTS is a general uncertainty quantification algorithm that can be applied to any multivariate multi-step forecaster, with statistical guarantees of validity. • CopulaCPTS produces significantly sharper and more calibrated uncertainty estimates than state-of-the-art baselines on 4 benchmark datasets, both synthetic and real.

