TEMPORAL DEPENDENCIES IN FEATURE IMPORTANCE FOR TIME SERIES PREDICTION

Abstract

Time series data introduces two key challenges for explainability methods: firstly, observations of the same feature over subsequent time steps are not independent, and secondly, the same feature can have varying importance to model predictions over time. In this paper, we propose Windowed Feature Importance in Time (WinIT), a feature removal based explainability approach to address these issues. Unlike existing feature removal explanation methods, WinIT explicitly accounts for the temporal dependence between different observations of the same feature in the construction of its importance score. Furthermore, WinIT captures the varying importance of a feature over time, by summarizing its importance over a window of past time steps. We conduct an extensive empirical study on synthetic and real-world data, compare against a wide range of leading explainability methods, and explore the impact of various evaluation strategies. Our results show that WinIT achieves significant gains over existing methods, with more consistent performance across different evaluation metrics.

1. INTRODUCTION

Reliably explaining predictions of machine learning models is important given their wide-spread use. Explanations provide transparency and aid reliable decision making, especially in domains such as finance and healthcare, where explainability is often an ethical and legal requirement (Amann et al., 2020; Prenio & Yong, 2021) . Multivariate time series data is ubiquitous in these sensitive domains, however explaining time series models has been relatively under explored. In this work we focus on saliency methods, a common approach to explainability that provides explanations by highlighting the importance of input features to model predictions (Baehrens et al., 2010; Mohseni et al., 2020) . It has been shown that standard saliency methods underperform on deep learning models used in the time series domain (Ismail et al., 2020) . In time series data, observations of the same feature at different points in time are typically related and their order matters. Methods that aim to highlight important observations but treat them as independent face significant limitations. Furthermore, it is important to note that the same observation of a feature can vary in importance to predictions at different times, which we refer to as the temporal dependency in feature importance. For example, there can be a delay between important feature shifts and a change in the model's predictions. These temporal dynamics can be difficult for current explainability methods to capture. To address these challenges, we propose the Windowed Feature Importance in Time (WinIT), a feature removal based method which determines the importance of a given observation to the predictions over a time window. Feature removal-based methods generate importance by measuring the change in outcomes when a particular feature is removed. However, removing a feature at a particular time does not account for the dependence between current and future observations of the same feature. WinIT addresses this by assigning the importance of a feature at a specific time based on a difference of two scores, that each take the temporal dependence of the subsequent observations into account. To capture the varying importance of the same feature observation to predictions at different times, WinIT aggregates the importance to predictions over a window of time steps to generate its final score. This allows our approach to identify important features that have delayed impact on the predictive distribution, thus better capturing temporal dynamics in feature importance. It is well known that the evaluation of explainability methods is challenging because no ground truth identification of important features exists for real world data (Doshi-Velez & Kim, 2017) . A

