Ablation Path Saliency

Abstract

We consider the saliency problem for black-box classification. In image classification, this means highlighting the part of the image that is most relevant for the current decision. We cast the saliency problem as finding an optimal ablation path between two images. An ablation path consists of a sequence of ever smaller masks, joining the current image to a reference image in another decision region. The optimal path will stay as long as possible in the current decision region. This approach extends the ablation tests in Sturmfels et al. (2020). The gradient of the corresponding objective function is closely related to the integrated gradient method Sundararajan et al. ( 2017). In the saturated case (when the classifier outputs a binary value) our method would reduce to the meaningful perturbation approach Fong & Vedaldi (2017), since crossing the decision boundary as late as possible would then be equivalent to finding the smallest possible mask lying on the decision boundary. Our interpretation provides geometric understanding of existing saliency methods, and suggests a novel approach based on ablation path optimisation.

1. Introduction

The basic idea of saliency or attribution is to provide something from which a human can judge how a classifier arrived at its decision of the prediction it gives for a certain input. It is difficult to give a more mathematical definition, but various properties that such a method should fulfill have been proposed. Sundararajan et al. (2017) give axioms, of which sensitivity comes closest to the notion of saliency. Essentially, the features on which the output is most sensitive should be given a higher saliency value. The authors give further axioms to narrow it down -implementation invariance, completeness, linearity and symmetry-preservation -and obtain a corresponding method: the integrated gradient method. Note that we have another way to arrive at a similar method, see §4.1 Fong & Vedaldi ( 2017) is closer to our work: the authors directly compute the saliency of a given pixel by deleting, or altering that pixel, to see how this affects the output of the classifier. Our method is to define a proper maximisation problem as follows. First, we define ablation paths as time dependent smooth masks φ : [0, 1] → C(Ω, R), going a full mask to an empty mask, such that at each pixel the mask value decreases over time (see Figure 1 ). We also impose constant area speed: 



Figure 1: Example of how an ablation path (sequence of masks, top) gives rise to a transition between a current target (image of a house finch) and a baseline (orca whale).

