MULTIWAVE: MULTIRESOLUTION DEEP ARCHITEC-TURES THROUGH WAVELET DECOMPOSITION FOR MUL-TIVARIATE TIMESERIES FORECASTING AND PREDIC-TION

Abstract

One of the challenges in multivariate time series modeling is that changes in signals occur with different frequencies, even when the sampling rate is consistent across signals. In the case of multivariate time series prediction, the outcome is also determined by patterns of different frequencies. These encapsulate both long-term and short-term effects, which have so far not been sufficiently leveraged by deep learning time series models. We fill this gap by introducing a framework, called MultiWave, which augments any deep learning time series model with components operating at the intrinsic frequencies of the signals. MultiWave applies wavelet decomposition on each signal to obtain subsignals of different frequencies and groups all subsignals in the same frequency band together to train a component. The output of the components is combined through a gating mechanism that removes irrelevant frequencies for the given predictive task. We show that MultiWave accurately determines the informative frequency bands and that the augmented models including components trained to operate on those bands outperform the original models. We further show that applying MultiWave on top of different deep learning models improves their performance in several real-world applications.

1. INTRODUCTION

Multivariate time series prediction has long been a crucial task in machine learning, as it has important applications in many fields such as healthcare, traffic flow, and economic forecasting. However, the final prediction in these applications can depend on many factors, such as information at different frequencies, long-term and short-term changes in input signals. Moreover, in many tasks, observations come from multiple sources and are often collected at various sampling rates. Here, we propose a model-agnostic approach that can leverage temporal dependencies at different frequencies and scales in multivariate time series data that might be collected with multiple sampling rates (multirate time series data) using multilevel discrete wavelet decomposition. There are two important categories of methods for time series analysis: Time-domain methods that consider the time series as a sequence of ordered points in time and frequency-domain methods that use transform algorithms such as Fourier transform and Z-transform to analyze the original sequence in the frequency spectrum. Deep learning-based methods that are introduced into time series analysis, such as recurrent neural networks (Williams & Zipser, 1989) , Convolutional Neural Networks (CNN) (Zheng et al., 2016) and more recently transformers (Wen et al., 2022) achieve state-of-the-art results in many applications (Lai et al., 2022; Tipirneni & Reddy, 2021; Huang et al., 2022) . However, they have two notable shortcomings in handling multivariate time series data. To overcome these deficiencies, we propose a novel model-agnostic framework, which uses discrete wavelet decomposition to break signals into different frequency components (subsignals), group



First, most of these methods only use information available in the time domain and cannot leverage information present in the frequency domain of the signals. Additionally, these methods cannot directly model signals that are collected with different frequencies (multirate signals), and upsampling or downsampling these time series to a single rate can artificially introduce or remove some important temporal dependencies Che et al. (2018a); Tipirneni & Reddy (2021); Che et al. (2018b).

