UNDERSTANDING AND MITIGATING ACCURACY DIS-PARITY IN REGRESSION

Abstract

With the widespread deployment of large-scale prediction systems in high-stakes domains, e.g., face recognition, criminal justice, etc., disparity on prediction accuracy between different demographic subgroups has called for fundamental understanding on the source of such disparity and algorithmic intervention to mitigate it. In this paper, we study the accuracy disparity problem in regression. To begin with, we first propose an error decomposition theorem, which decomposes the accuracy disparity into the distance between label populations and the distance between conditional representations, to help explain why such accuracy disparity appears in practice. Motivated by this error decomposition and the general idea of distribution alignment with statistical distances, we then propose an algorithm to reduce this disparity, and analyze its game-theoretic optima of the proposed objective function. We conduct experiments on four real-world datasets. The experimental results suggest that our proposed algorithms can effectively mitigate accuracy disparity while maintaining the predictive power of the regression models.

1. INTRODUCTION

Recent progress in machine learning has led to its widespread use in many high-stakes domains, such as criminal justice, healthcare, student loan approval, and hiring. Meanwhile, it has also been widely observed that accuracy disparity could occur inadvertently under various scenarios in practice (Barocas & Selbst, 2016) . For example, errors are inclined to occur for individuals of certain underrepresented demographic groups (Kim, 2016) . In other cases, Buolamwini & Gebru (2018) showed that notable accuracy disparity gaps exist across different racial and gender demographic subgroups on several real-world image classification systems. Moreover, Bagdasaryan et al. (2019) found out that a differentially private model even enlarges such accuracy disparity gaps. Such accuracy disparity gaps across demographic subgroups not only raise concerns in high-stake applications but also can be utilized by malicious parties causing information leakage (Yaghini et al., 2019) . Despite the ample needs of accuracy parity, most prior work limits its scope to studying the problem in binary classification settings (Hardt et al., 2016; Zafar et al., 2017b; Zhao et al., 2019; Jiang et al., 2019) . In a seminal work, Chen et al. ( 2018) analyzed the impact of data collection on accuracy disparity in general learning models. They provided a descriptive analysis of such parity gaps and advocated for collecting more training examples and introducing more predictive variables. While such a suggestion is feasible in applications where data collection and labeling is cheap, it is not applicable in domains where it is time-consuming, expensive, or even infeasible to collect more data, e.g., in autonomous driving, education, etc. Our Contributions In this paper, we provide a prescriptive analysis of accuracy disparity and aim at providing algorithmic interventions to reduce the disparity gap between different demographic subgroups in the regression setting. To start with, we first formally characterize why accuracy disparity appears in regression problems by depicting the feasible region of the underlying group-wise errors. We also provide a lower bound on the joint error and a complementary upper bound on the error gap across groups. Based on these results, we illustrate why regression models aiming to minimize the global loss will inevitably lead to accuracy disparity if the input distributions or decision functions differ across groups (see Figure 1a ). We further propose an error decomposition theorem that decomposes the accuracy disparity into the distance between the label populations and the distance between conditional representations. To mitigate such disparities, we propose two algorithms to reduce accuracy disparity via joint distribution alignment with total variation distance and Wasserstein distance, respectively. Furthermore, we analyze the game-theoretic optima of the objective function and illustrate the principle of our algorithms from a game-theoretic perspective (see Figure 1b ). To corroborate the effectiveness of our proposed algorithms in reducing accuracy disparity, we conduct experiments on four real-world datasets. Experimental results suggest that our proposed algorithms help to mitigate accuracy disparity while maintaining the predictive power of the regression models. We believe our theoretical results contribute to the understanding of why accuracy disparity occurs in machine learning models, and the proposed algorithms provides an alternative for intervention in real-world scenarios where accuracy parity is desired but collecting more data/features is time-consuming or infeasible.

2. PRELIMINARIES

Notation We use X ⊆ R d and Y ⊆ R to denote the input and output space. We use X and Y to denote random variables which take values in X and Y, respectively. Lower case letters x and y denote the instantiation of X and Y . We use H(X) to denote the Shannon entropy of random variable X, H(X | Y ) to denote the conditional entropy of X given Y , and I(X; Y ) to denote the mutual information between X and Y . To simplify the presentation, we use A ∈ {0, 1} as the sensitive attribute, e.g., gender, race, etc. Let H be the hypothesis class of regression models. In other words, for h ∈ H, h : X → Y is a predictor. Note that even if the predictor does not explicitly take the sensitive attribute A as an input variable, the prediction can still be biased due to the correlations with other input variables. In this work we study the stochastic setting where there is a joint distribution D over X, Y and A from which the data are sampled. For a ∈ {0, 1} and y ∈ R, we use D a to denote the conditional distribution of D given A = a and D y to denote the conditional distribution of D given Y = y. For an event E, D(E) denotes the probability of E under D. Given a feature transformation function g : X → Z that maps instances from the input space X to feature space Z, we define g D := D • g -1 to be the induced (pushforward) distribution of D under g, i.e., for any event  E ⊆ Z, g D(E ) := D({x ∈ X | g(x) ∈ E }). D (h) := E D [(Y - h(X)) 2 ]. To make the notation more compact, we may drop the subscript D when it is clear from the context. Furthermore, we also use MSE D ( Y , Y ) to denote the mean squared loss between the predicted variable Y = h(X) and the true label Y over the joint distribution D. Similarly, we also use CE D (A A) denote the cross-entropy loss between the predicted variable A and the true label A over the joint distribution D. Throughout the paper, we make the following standard assumption in regression problems: Assumption 2.1. There exists M > 0, such that for any hypothesis H h : X → Y, h ∞ ≤ M and |Y | ≤ M . Problem Setup We study the fair regression problem: the goal is to learn a regressor that is fair in the sense that the errors of the regressor are approximately equal across the groups given by the sensitive attribute A. We assume that the sensitive attribute A is only available to the learner during



Game-theoretic illustration of our algorithms.

Figure 1: The left figure illustrates how accuracy disparity arises by minimizing the global squared loss. The right figure gives a schematic illustration of the proposed algorithmic framework.

We use (•) + to indicate the value of a variable remains unchanged if it is positive or otherwise 0, i.e., (Y ) + equals to Y if the value of Y is positive or otherwise 0. Given a joint distribution D, the error of a predictor h under D is defined as Err

