ROBUST MULTIVARIATE TIME-SERIES FORECASTING: ADVERSARIAL ATTACKS AND DEFENSE MECHANISMS

Abstract

This work studies the threats of adversarial attack on multivariate probabilistic forecasting models and viable defense mechanisms. Our studies discover a new attack pattern that negatively impact the forecasting of a target time series via making strategic, sparse (imperceptible) modifications to the past observations of a small number of other time series. To mitigate the impact of such attack, we have developed two defense strategies. First, we extend a previously developed randomized smoothing technique in classification to multivariate forecasting scenarios. Second, we develop an adversarial training algorithm that learns to create adversarial examples and at the same time optimizes the forecasting model to improve its robustness against such adversarial simulation. Extensive experiments on real-world datasets confirm that our attack schemes are powerful and our defense algorithms are more effective compared with baseline defense mechanisms.

1. INTRODUCTION

Understanding the robustness for time-series models has been a long-standing issue with applications across many disciplines such as climate change (Mudelsee, 2019) , financial market analysis (Andersen et al., 2005; Hallac et al., 2017) , down-stream decision systems in retail (Böse et al., 2017) , resource planning for cloud computing (Park et al., 2019; 2020) , and optimal control of vehicles (Kim et al., 2020) . In particular, the notion of robustness defines how sensitive the model output is when authentic data is (potentially) perturbed with noises. In practice, as observation data are often corrupted by measurement noises, it is important to develop statistical forecasting models that are less sensitive to such noises (Brown, 1957; Brockwell & Davis, 2009; Taylor & Letham, 2018) or more stable against outliers that might arise from such corruption (Connor et al., 1994; Gelper et al., 2010; Liu & Zhang, 2021; Wang & Tsay, 2021) . However, these approaches have not considered the possibility of adversarial noises which are strategically created to mislead the model rather than being sampled from a known distribution. As a matter of fact, vulnerabilities against such adversarial noises have been previously pointed out (Szegedy et al., 2013; Goodfellow et al., 2014b) in classification. In practice, it has been shown that human-imperceptible adversarial perturbation can alter classification outcomes of a deep learning (DL) model, revealing a severe threat to many safety-critical systems . As such a risk is associated with the high capacity to fit complex data pattern of DL, we postulate that similar threats might also occur in forecasting where modern DL-based forecasting models (Rangapuram et al., 2018; Salinas et al., 2020; Lim et al., 2020; Wang et al., 2019; Park et al., 2022) have become the dominant approach. For example, to mislead the forecasting of a particular stock, the adversaries might attempt to alter some features external to the stock's financial valuation to maximize the gap between predictions of its values on authentic and altered features. The feasibility of such an adversarial attack has been recently demonstrated with tweet messages (Xie et al., 2022) on a text-based stock forecasting. Motivated by these real scenarios, we propose to investigate such adversarial threats on more practical forecasting models whose predictions are based on more precise features, e.g. valuations of other stock indices. Intuitively, rather than releasing adverse information to alter the sentiment about the target stock on social media, the adversaries can instead invest hence change the valuation adversely for a selected subset of stock indices (not including the target stock) which is arguably harder to detect. Interestingly, despite being seemingly plausible given the vast literature on adversarial attack for classification models, formulating such imperceptible attack under a multivariate forecasting setup is not straightforward. This is due to several differences between forecasting and classification, particularly in terms of unique characteristic of time series, e.g., multi-step predictions, correlation over multiple time series, and probabilistic predictions. These differences open up the question of how adversarial perturbations and robustness should be defined more properly in time series setting. Although there have been a few recent studies in this direction based on randomized smoothing (Yoon et al., 2022) , these approaches are all restricted to univariate forecasting where the attack has to make adverse alterations directly to the target time series. Thus, under the less studied scenario of multivariate time-series forecasting setup, it remains unclear whether the attack to a target time series can be made instead via perturbing the other correlated time series; and whether it is defensible against such adversarial threats. In particular, as illustrated above in the stock forecasting example, there are new regimes of sparse and indirect cross time series attack under multivariate time-series scenarios, which are more effective and realistic than the direct attack in univariate cases. In order to understand whether such new regimes of attack exists and can be defended against, we raise three questions: 1. Indirect Attack. Can we mislead the prediction of some target time series via perturbations on the other time series? 2. Sparse Attack. Can such perturbations be sparse and non-deterministic to be less perceptible? 3. Robust Defense. Can we defend against those indirect and imperceptible attacks? Here we summarize our technical contributions by answering the questions above: Regarding indirect attack, we provide general framework of adversarial attack in multivariate time series (see Section 3.1). Then, we devise a deterministic attack (see Section 3.2) to the state-ofthe-art probabilistic multivariate forecasting model. The attack changes the model's prediction on the target time series via adversely perturbing a subset of other time series. This is achieved via formulating the perturbation as solution of an optimization task with packing constraints. Regarding sparse attack, we develop a non-deterministic attack (see Section 3.3) that adversely perturbs a stochastic subset of time series related to the target time series, which makes the attack less perceptible. This is achieved via a stochastic and continuous relaxation of the above packing constraint which are shown (see Section 5) to be more effective than the deterministic attack in certain cases. Moreover, unlike deterministic attack, its differentiability makes it suitable to be directly integrated as part of a differentiable defense mechanism that can be optimized via gradient descent in an end-to-end fashion, as discussed later in Section 4.2. Regarding robust defense, we propose two defense mechanisms. First, we adapt randomized smoothing to the new multivariate forecasting setup with robust certificate. Second, we devise a defense mechanism (see Section 4.2) via solving a mini-max optimization task which minimizes the maximum expected damage caused by the probabilistic attack that continually updates the generation of its adverse perturbations in response to the model updates. Their effectiveness are demonstrated across extensive experiments in Section 5. Furthermore, our experiments in Section 5.3 demonstrate that attacks designed for univariate cases cannot be reused as an effective attack to multivariate forecasting models, which highlights the importance and novelty of our studies. The code to reproduce our experiments results can be found at https://github.com/awslabs/gluonts/tree/dev/src/gluonts/ nursery/robust-mts-attack. 



Deep Forecasting Models. The recent decades have witnessed a tremendous progress in DNNbased forecasting models. Given the temporal dependency of time series data, RNN and CNN-based architectures have been proved a success for time series forecasting tasks, see Rangapuram et al. (2018); Lim et al. (2020); Wang et al. (2019); Salinas et al. (2020) and Oord et al. (2016); Bai et al.

