ROCOURSENET: ROBUST TRAINING OF A PREDICTION AWARE RECOURSE MODEL

Abstract

Counterfactual (CF) explanations for machine learning (ML) models are preferred by end-users, as they explain the predictions of ML models by providing a recourse (or contrastive) case to individuals who are adversely impacted by predicted outcomes. Existing CF explanation methods generate recourses under the assumption that the underlying target ML model remains stationary over time. However, due to commonly occurring distributional shifts in training data, ML models constantly get updated in practice, which might render previously generated recourses invalid and diminish end-users trust in our algorithmic framework. To address this problem, we propose RoCourseNet, a training framework that jointly optimizes for predictions and recourses that are robust to future data shifts. We have three main contributions: (i) We propose a novel virtual data shift (VDS) algorithm to find worst-case shifted ML models by explicitly considering the worst-case data shift in the training dataset. (ii) We leverage adversarial training to solve a novel tri-level optimization problem inside RoCourseNet, which simultaneously generates predictions and corresponding robust recourses. (iii) Finally, we evaluate RoCourseNet's performance on three real-world datasets and show that RoCourseNet outperforms state-of-the-art baselines by ∼10% in generating robust CF explanations.

1. INTRODUCTION

To explain the prediction made by a Machine Learning (ML) model on data point x, counterfactual (CF) explanation methods find a new counterfactual example x cf , which is similar to x but gets a different/opposite prediction from the ML model. CF explanations (Wachter et al., 2017; Karimi et al., 2020; Verma et al., 2020) are often preferred by end-users as they provide actionable recoursefoot_0 to individuals who are negatively impacted by algorithm-mediated decisions. For example, CF explanation techniques can provide recourse for impoverished loan applicants whose loans have been denied by a bank's ML algorithm. Most CF explanation techniques assume that the underlying ML model is stationary and does not change over time (Barocas et al., 2020) . However, in practice, ML models are often updated regularly when new data is available to improve predictive accuracy on the new shifted data distribution. This shifted ML model might render previously recommended recourses ineffective (Rawal et al., 2020) , and in turn, diminish end users' trust towards our system. For example, when providing a recourse to a loan applicant who was denied a loan by the bank's ML algorithm, it is critical to approve re-applications that fully follow recourse recommendations, even if the bank updates their ML model in the meantime. This necessitates the development of robust algorithms that can generate recourses which remain effective (or valid) for an end-user in the face of ML models being frequently updated. Figure 1 illustrates this challenge of generating robust recourses. Limitations of Prior Work. To our knowledge, only two studies (Upadhyay et al., 2021; Nguyen et al., 2022) propose algorithmic methods to generate robust recourses. Unfortunately, both these studies suffer from two major limitations. First, both methods are based on strong modeling assumptions which degrades their effectiveness at finding robust recourses (as we show in Section 4). For example, Upadhyay et al. ( 2021 locally approximated via a linear function, and adopt LIME (Ribeiro et al., 2016) to find this linear approximation. However, recent works show that the local approximation generated from LIME is unfaithful (Laugel et al., 2018; Rudin, 2019) and inconsistent (Slack et al., 2020; Alvarez-Melis & Jaakkola, 2018) . Similarly, Nguyen et al. ( 2022) assumes that the underlying data distribution can be approximated using kernel density estimators (Bickel et al., 2009) . However, kernel density estimators suffers from the curse of dimensionality (Bellman, 1961), i.e., they perform exponentially worse with increasing dimensionality of data (Crabbe, 2013; Nagler & Czado, 2016) , which limits its usability in estimating the data distributions of real-world high-dimensional data. Second, these two techniques are post-hoc methods designed for use with proprietary black-box ML models whose training data and model weights are not available. However, with the advent of data regulations that enshrine the "Right to Explanation" (e.g., EU-GDPR (Wachter et al., 2017)), service providers are required by law to communicate both the decision outcome (i.e., the ML model's prediction) and its actionable implications (i.e., a recourse for this prediction) to an end-user. In these scenarios, the post-hoc assumption is overly limiting, as service providers can build recourse models that leverage the knowledge of their ML model to generate higher-quality recourses. In fact, prior work (Guo et al., 2021) has shown that post-hoc CF explanation approaches are unable to balance the cost-invalidity trade-off (Rawal et al., 2020) , which is an important consideration in generating recourses. To date, very little prior work departs from the post-hoc paradigm; Guo et al. (2021) propose one such approach, unfortunately, it does not consider the robustness of generated recourses. Contributions. We propose Robust ReCourse Neural Network (or RoCourseNet), a novel algorithmic framework for generating recourses which: (i) departs from the prevalent post-hoc paradigm of generating recourses; while (ii) explicitly optimizing the robustness of the generated recourses. RoCourseNet makes the following three novel contributions: • (Formulation-wise) We formulate the robust recourse generation problem as a tri-level (min-maxmin) optimization problem, which consists of two sub-problems: (i) a bi-level (max-min) problem which simulates a worst-case attacker to find an adversarially shifted model by explicitly simulating the worst-case data shift in the training dataset; and (ii) an outer minimization problem which simulates an ML model designer who wants to generate robust recourses against this worst-case bi-level attacker. Unlike prior approaches, our bi-level attacker formulation explicitly connects shifts in the underlying data distribution to corresponding shifts in the ML model parameters. 



Note that counterfactual explanation(Wachter et al., 2017) and algorithmic recourse(Ustun et al., 2019) are closely related(Verma et al., 2020; Stepin et al., 2021). Hence, we use these terms interchangeably.



) assume that the ML model's decision boundary can be 𝑓The decision boundary of model f (•, θ) trained on the original data, and a recourse x cf of the example x generated CF explanation methods. Updated decision boundary of retrained new model f (•, θ ′ ) with newly available data (or a shifted data distribution). Under the shifts in data distribution and model, the recourse x cf becomes invalid, but xcf is valid. We call xcf as a robust recourse.

Figure 1: Illustration of the robust recourse generation. (a) Given an input data point x, CF explanation methods generate a new recourse x cf which lies on the opposite side of decision boundary f (.; θ). (b) As new data is made available, the ML model's decision boundary is updated as f (.; θ ′ ). This shifted decision boundary f (.; θ ′ ) invalidates the chosen recourse x cf (as x and x cf lie on the same side of the shifted model f (.; θ ′ )). (c) However, robust CF explanation methods generate a robust recourse xcf for input x by anticipating the future shifted model f (.; θ ′ ).

• (Methodology-wise) We propose RoCourseNet for solving our tri-level optimization problem for generating robust recourses. RoCourseNet relies on two key ideas: (i) we propose a novel Virtual Data Shift (VDS) algorithm to optimize for the inner bi-level (max-min) attacker problem, which results in an adversarially shifted model; and (ii) inspired by Guo et al. (2021), RoCourseNet leverages a block-wise coordinate descent training procedure to optimize the robustness of generated

