SEQSHAP: SUBSEQUENCE LEVEL SHAPLEY VALUE EXPLANATIONS FOR SEQUENTIAL PREDICTIONS

Abstract

With the increasing demands of interpretability in real-world applications, various methods for explainable artificial intelligence (XAI) have been proposed. However, most of them overlook the interpretability in sequential scenarios, which have a wide range of applications, e.g., online transactions and sequential recommendations. In this paper, we propose a Shapley value based explainer named SeqSHAP to explain the model predictions in sequential scenarios. Compared to existing methods, SeqSHAP provides more intuitive explanations at a subsequence level, which explicitly models the effect of contextual information among the related elements in a sequence. We propose to calculate subsequence-level feature attributions instead of element-wise attributions to utilize the information embedded in sequence structure, and provide a distribution-based segmentation method to obtain reasonable subsequences. Extensive experiments on two online transaction datasets from a real-world e-commerce platform show that the proposed method could provide valid and reliable explanations for sequential predictions.

1. INTRODUCTION

Sequential prediction tasks have a wide range of applications in real-world, e.g., Online Transaction (Wang et al., 2017; Zhang et al., 2018; Weber et al., 2018; Tam et al., 2019; Zhu et al., 2020; Chen & Lai, 2021) and Sequential Recommendation (Quadrana et al., 2017; Tang & Wang, 2018; Sun et al., 2019; Shen et al., 2021; Cui et al., 2022) , since sequences contain continuous signals which are important for model predictions. With the development of deep learning technique, sequence-based models have achieved a desirable performance in recent years (Hidasi et al., 2015; Quadrana et al., 2017; Wang et al., 2017; Tang & Wang, 2018; Zhang et al., 2018; Sun et al., 2019; Zhu et al., 2020; Qiao & Wang, 2022) . However, the complicated sequential data and increased model complexity make it hard for humans to understand the prediction of models. Indeed, for security and trust considerations, it is essential to develop effective explainable artificial intelligence (XAI) methods for sequence-based models in scenarios like fraud detection and medical care, so that end-users could understand how model predictions are produced with these complicated sequential data and models. In recent years, considerable efforts have been made on the model explanation algorithms (Ribeiro et al., 2016; Shrikumar et al., 2017; Lundberg & Lee, 2017; Selvaraju et al., 2017; Wachter et al., 2017; Alvarez-Melis & Jaakkola, 2018; Mothilal et al., 2020; Slack et al., 2021; Ghalebikesabi et al., 2021; Ali et al., 2022) . Among these works, feature attribution methods (Ribeiro et al., 2016; Shrikumar et al., 2016; 2017; Lundberg & Lee, 2017) are a popular family of post-hoc XAI methods. They calculate an attribution score for each feature to capture those important features for model predictions. However, most existing methods mainly pay attention to explain tabular data or images. And when dealing with the data and models in sequential scenarios, the complex input sequences make the element-wise explanations produced by these methods less explainable. The high-dimensional features and abundant interactions bring difficulty to existing element-wise XAI methods to provide explanations. Separately assigning attribution scores to individual feature cells in the sequence is not informative enough for users to understand the predictions. In addition, the great amount of features in a sequence could bring an extensive execution cost for existing methods, since the time complexity of them are mostly related to the number of features to be explained.

