SPECTRANET: MULTIVARIATE FORECASTING AND IM-PUTATION UNDER DISTRIBUTION SHIFTS AND MISSING DATA

Abstract

In this work, we tackle two widespread challenges in real applications for timeseries forecasting that have been largely understudied: distribution shifts and missing data. We propose SpectraNet, a novel multivariate time-series forecasting model that dynamically infers a latent space spectral decomposition to capture current temporal dynamics and correlations on the recent observed history. A Convolution Neural Network maps the learned representation by sequentially mixing its components and refining the output. Our proposed approach can simultaneously produce forecasts and interpolate past observations and can, therefore, greatly simplify production systems by unifying imputation and forecasting tasks into a single model. SpectraNetachieves SoTA performance simultaneously on both tasks on five benchmark datasets, compared to forecasting and imputation models, with up to 92% fewer parameters and comparable training times. On settings with up to 80% missing data, SpectraNethas average performance improvements of almost 50% over the second-best alternative.

1. INTRODUCTION

Multivariate time-series forecasting is an essential task in a wide range of domains. Forecasts are a key input to optimize the production and distribution of goods (Böse et al., 2017) , predict healthcare patient outcomes (Chen et al., 2015) , plan electricity production (Olivares et al., 2022) , build financial portfolios (Emerson et al., 2019) , among other examples. Due to its high potential benefits, researchers have dedicated many efforts to improving the capabilities of forecasting models, with breakthroughs in model architectures and performance (Benidis et al., 2022) . The main focus of research in multivariate forecasting has been on accuracy and scalability, to which the present paper contributes. In addition, we identify two widespread challenges for real applications which have been largely understudied: distribution shifts and missing data. We refer to distribution shifts as changes in the time-series behavior. In particular, we focus on discrepancies in distribution between the train and test data, which can considerably degrade the accuracy (Kuznetsov & Mohri, 2014; Du et al., 2021; Xu et al., 2022; Ivanovic et al., 2022) . This has become an increasing problem in recent years with the COVID-19 pandemic, which disrupted all aspects of human activities. Missing values is a generalized problem. Some common causes include faulty sensors, the impossibility of gathering data, and misplacement of information. As we demonstrate in our experiments, these challenges hinder the performance of current state-of-the-art (SoTA), limiting their use in applications where these problems are predominant. In this work, we propose SpectraNet, a novel multivariate forecasting model that achieves SoTA performance in benchmark datasets and is also intrinsically robust to distribution shifts and extreme cases of missing data. SpectraNet achieves its high accuracy and robustness by dynamically inferring a latent vector projected on a temporal basis, a process we name latent space spectral decomposition (LSSD). A series of convolution layers then synthesize both the reference window, which is used to infer the latent vectors and the forecast window. In practice, imputation models are first used to fill the missing information for all downstream tasks, including forecasting. SpectraNet can greatly simplify production systems by unifying imputation and forecasting tasks into a single model. The main contributions are: • Latent Vector Inference: methodology to dynamically capture current dynamics of the target time-series into a latent space, replacing parametric encoders. • Latent Space Spectral Decomposition: representation of a multivariate time-series window on a shared latent space with temporal dynamics. • SpectraNet: novel multivariate forecasting model that simultaneously imputes missing data and forecasts future values, with SoTA performance on several benchmark datasets and demonstrated robustness to distribution shifts and missing values. We will make our code publicly available upon acceptance. The remainder of this paper is structured as follows. Section 2 introduces notation and the problem definition, Section 3 presents our method, Section 4 describes and presents our empirical findings. Finally, Section 5 concludes the paper. The literature review is included in A.1.

2. NOTATION AND PROBLEM DEFINITION

We introduce a new notation that we believe is lighter than the standard notation while being intuitive and formally correct. Let Y ∈ R M ×T be a multivariate time-series with M features and T timestamps. Let Y a:b ∈ R M ×(b-a) be the observed values for the interval [a, b), that is, Y 0:t is the set of t observations of Y from timestamp 0 to timestamp t -1 while Y t:t+H is the set of H observations of Y from timestamp t to timestamp t + H -1. Let y m,t ∈ R be the value of feature m at timestamp t. In this work we consider the multivariate point forecasting task, which consists of predicting the future values of a multivariate time-series sequence based on past observations. The main task of a model F Θ with parameters Θ at a timestamp t, is to produce forecasts for the future H values, denoted by Ŷt:t+H , based on the previous history Y 0:t .



To the best of our knowledge, SpectraNet is also the first solution that can simultaneously forecast the future values of a multivariate time series and accurately impute the past missing data.



Figure 1: SpectraNet architecture. The Latent Space Spectral Decomposition (LSSD) encodes shared temporal dynamics of the target window into Fourier waves and polynomial functions. Latent vector z is inferred with Gradient Descent minimizing reconstruction error on the reference window. The Convolution Network (CNN) generates the time-series window by sequentially mixing the components of the embedding and refining the output.

