BEYOND TRIVIAL COUNTERFACTUAL GENERATIONS WITH DIVERSE VALUABLE EXPLANATIONS

Abstract

Explainability of machine learning models has gained considerable attention within our research community given the importance of deploying more reliable machine-learning systems. Explanability can also be helpful for model debugging. In computer vision applications, most methods explain models by displaying the regions in the input image that they focus on for their prediction, but it is difficult to improve models based on these explanations since they do not indicate why the model fail. Counterfactual methods, on the other hand, indicate how to perturb the input to change the model prediction, providing details about the model's decision-making. Unfortunately, current counterfactual methods make ambiguous interpretations as they combine multiple biases of the model and the data in a single counterfactual interpretation of the model's decision. Moreover, these methods tend to generate trivial counterfactuals about the model's decision, as they often suggest to exaggerate or remove the presence of the attribute being classified. Trivial counterfactuals are usually not valuable, since the information they provide is often already known to the system's designer. In this work, we propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss to uncover multiple valuable explanations about the model's prediction. Further, we introduce a mechanism to prevent the model from producing trivial explanations. Experiments on CelebA and Synbols demonstrate that our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods. We will make the code public.

1. INTRODUCTION

Consider a face authentication system for unlocking a device. In case of non-authentications (possible false-negative predictions), this system could provide generic advices to its user such as "face the camera" or "remove any face occlusions". However, these may not explain the reason for the possible malfunction. To provide more insights regarding its decisions, the system could instead provide information specific to the captured image (its input data). It might list the input features that most contributed to its decision (e.g., a region of the input image), but this feature could be "face", which is trivial and does not suggest an alternative action to its user. Further, it provides little useful information about the model. Instead, non-trivial explanations may be key for better understanding and diagnosing the system-including the data it was trained on-and improving its reliability. Such explanations might improve systems across a wide variety of domains including in medical imaging [58] , automated driving systems [48] , and quality control in manufacturing [22] . The explainability literature aims to understand the decisions made by a machine learning (ML) model such as the aformentionned face authentication system. Counterfactual explanation methods [11, 13, 4] can help discover the limitations of a ML model by uncovering data and model biases. The counterfactual explanation methods provide perturbed versions of the input data that emphasize features that contributed most to the ML model's output. For example, if an authentication system is not recognizing a user wearing sunglasses then the system could generate an alternative image of the user's face without sunglasses that would be correctly recognized. This is different from other types of explainability methods such as feature importance methods [50, 51, 4] and boundary approximation methods [47, 37] . The former highlight salient regions of the input but do not indicate how the ML model could achieve a different prediction. The second family of methods produce

