MULTIVARIATE TIME-SERIES IMPUTATION WITH DIS-ENTANGLED TEMPORAL REPRESENTATIONS

Abstract

Multivariate time series often faces the problem of missing value. Many time series imputation methods have been developed in literature. However, they all rely on an entangled representation to model dynamics of time series, which may fail to fully exploit the multiple factors (e.g., periodic patterns) presented in the data. Moreover, the entangled representations usually have no semantic meaning, and thus they often lack interpretability. In addition, many recent models are proposed to deal with the whole time series to identify temporal dynamics, but they are not scalable to long time series. Different from existing approaches, we propose TIDER, a novel matrix factorization-based method with disentangled temporal representations that account for multiple factors, namely trend, seasonality, and local bias, to model complex dynamics. The learned disentanglement makes the imputation process more reliable and offers explainability for imputation results. Moreover, TIDER is scalable to long time series. Empirical results show that our method outperforms existing approaches on three typical real-world datasets, especially on long time series, reducing mean absolute error by up to 50%. It also scales well to long datasets on which existing deep learning based methods struggle. Disentanglement validation experiments further highlight the robustness and accuracy of our model.

1. INTRODUCTION

Multivariate time series analysis (e.g., forecasting (Zeng et al., 2021) and classification (Li et al., 2022) ) has a wide spectrum of applications like traffic flow forecasting (Liu et al., 2020) , electricity demand prediction (Kaur et al., 2021) , motion detection (Laddha et al., 2021) , health monitoring (Tonekaboni et al., 2021) , etc. Most of these multivariate time series analysis approaches typically assume intact input for building models. However, real-world multivariate time series tends to have missing values caused by factors like device malfunction, communication failure, or costly measurement, leading to impaired performance of these approaches, or even rendering them inapplicable. In light of this, many time series imputation methods have been proposed to infer missing values from the observed ones. A multivariate time series, denoted as X ∈ R N ×T , consists of N univariate time series (called channels) spanning over T time steps. Hence, it offers two perspectives for imputation: modeling cross-channel correlations and exploiting temporal dynamics. Earlier methods (Batista et al., 2002; Acuna & Rodriguez, 2004; Box et al., 2015) either aggregate observed entries across channels by estimating similarity between distinct channels, or solely exploit local smoothness or linear assumption in the same channel to fill in missing values. Since these methods lack the ability of modeling nonlinear dynamics and complex correlations, they may not perform well in practice. In recent years, deep learning based methods (Liu et al., 2019; Tashiro et al., 2021; Cini et al., 2022) were proposed for time series imputation. These methods typically employ Recurrent Neural Networks (RNNs) as the backbone to jointly model nonlinear dynamics along temporal dimension (by updating hidden states nonlinearly over time steps) and cross-channel correlations (by mapping x t ∈ R N from data space to hidden state space). These RNN-based imputation models achieve better results by further combining with multi-task loss (Cao et al., 2018; Cini et al., 2022) or adversarial training (Liu et al., 2019; Miao et al., 2021) . Despite their success, these methods rely solely on a single entangled representation (hidden state) to model dynamics. However, for many real-world multivariate time series, their dynamics are rich combinations of multiple independent factors like trend, seasonality (periodicity), and local idiosyncrasy (Woo et al., 2022) . Modeling combinations of these factors with a single entangled representation may not give good performance, as the entangled representation has to compromise itself to explain multiple orthogonal patterns (like local changes or exogenous interventions vs global patterns) together (Bengio et al., 2013) . Furthermore, it becomes exacerbated when seasonal patterns dominate, as RNNs lack the inductive bias to initiatively capture periodicity (Hewamalage et al., 2021) . In addition, the hidden states that these models learned are entangled and complex combination of various components. Thus it is difficult for them to provide interpretable information to explain imputation. Moreover, these methods require the entire time series to be fed in to their models at each forward step, to capture temporal dynamics.This is prohibitively costly for large T . Therefore, these methods are not applicable to long datasets. To address these limitations, in this paper, we propose a novel multivariate time series imputation method, Time-series Imputation with Disentangled tEmporal Representations (TIDER), in which we explicitly model the complex dynamics of multivariate time series with disentangled representations to account for different factors. We employ a low-rank matrix decomposition framework, and achieve the disentanglement by imposing different forms of constraints on different representations which compose the low-rank matrix. In particular, we introduce a neighboring-smoothness representation matrix to explain the trend, a Fourier series-based representation matrix to define periodic inductive bias, and a local bias representation matrix to capture idiosyncrasies specific to individual time step. These disentangled representations offer greater flexibility and robustness to model the factors contributed to the dynamics in time series. Another notable benefit of these semantically meaningful disentangled representations is that they offer TIDER an interpretable perspective on imputation. Moreover, TIDER is a scalable model. It can be applied to time-series datasets with large T and achieve much better imputation performance. In summary, our contributions are as follows. • We propose TIDER, a new multivariate time series imputation model, which is featured with effective and explainable disentangled representations to account for various factors that characterize the complex dynamics of time series. To the best of our knowledge, TIDER is the first model to learn disentangled representations for multivariate time series imputation. • TIDER is the first imputation model that introduces a learnable Fourier series-based representation to capture periodic patterns inherent in time series data. • Extensive experiments show that our proposed method outperforms the baseline imputation methods in terms of effectiveness and scalability. Especially, for imputation task in long time series, TIDER achieves more than 50% improvement in MAE, compared with the best imputation baseline approach. Furthermore, TIDER is a scalable model. It can easily handle long multivariate time series while existing deep-learning methods struggle. We also demonstrate the explanability of the disentangled representations with several case studies.

2. RELATED WORK

Early time series imputation methods based on simple statistical strategies focus on exploiting local smoothness of temporal dimension as well as the similarity between different channels. For example, SimpleMean/SimpleMedian (Fung, 2006) imputes missing values by averaging, and KNN (Batista et al., 2002) aggregates cross-channel observations to fill in missing slots with k-nearest neighbors. Linear dynamics-based imputation methods, including linear imputation and state-space models (Durbin & Koopman, 2012) , have also been employed. MICE (Van Buuren & Groothuis-Oudshoorn, 2011) explores the idea of estimating missing slots with chain equations. These methods typically lack the ability of exploiting nonlinear dynamics and complex correlation across channels.

