THE COST OF PRIVACY IN FAIR MACHINE LEARNING Anonymous authors Paper under double-blind review

Abstract

A common task in fair machine learning is training ML models that preserve certain summary statistics across subpopulations defined by sensitive attributes. However, access to such sensitive attributes in training data is restricted and the learner must rely on noisy proxies for the sensitive attributes. In this paper, we study the effect of a privacy mechanism that obfuscates the sensitive attributes from the learner on the fairness of the resulting classifier. We show that the cost of privacy in fair ML is a decline in the generalizability of fairness constraints.

1. INTRODUCTION

The fairness of machine learning systems is gaining increasing attention in recent years. Among the numerous fairness objectives is ensuring that a machine learning model does not discriminate against subpopulations that are typically identified by sensitive attributes (e.g., race, gender). When training a fair model and evaluating model bias, it is necessary to possess sensitive attributes; however, access to and use of such sensitive data is frequently prohibited by laws and regulations. Credit card companies, for instance, are not permitted to inquire about a person's race when they apply for credit, yet they must demonstrate that their decisions are not discriminatory (Chen et al., 2019) . Ideally, sensitive personal information should not be disclosed during the training of ML models. However, it is impossible to ensure exact notions of fairness (such as demographic parity or equality of opportunity) without any knowledge of the sensitive data. Fortunately, differential privacy (Dwork et al., 2006 ) is a promising workaround, which can offer a graceful compromise between privacy and utility. Mozannar et al. (2020) propose to release sensitive attributes in a locally differentially private way: adding noise to the sensitive data so that adversaries cannot infer any information with high confidence about a single record. The advantage of the privacy mechanism proposed by Mozannar et al. ( 2020) is an invariance property: exact notions of fairness with regard to true sensitive attributes and noisy sensitive attributes are equivalent. An implication of the invariance property is that the optimal model of fairness can be learned at the population level. Nonetheless, it remains unclear what the precise statistical impact of privacy on fairness is. In this work, we study the statistical cost of privacy on fairness in the task of learning fair ML models with differentially private sensitive attributes. The main benefits of the developed theory are 1. statistically principled: We propose a statistically principled metric to characterize the cost of privacy on fairness. A restricted notion of statistical efficiency precisely quantifies the privacy cost asymptotically. 2. interpretable: Privacy leads to a decline in the statistical efficiency. Such efficiency loss is interpretable: it explicitly depends on the privacy budget, the subpopulation imbalance level, and few other problem-specific parameters. The rest of this paper is organized as follows. In Section 2, we formalize the problem setup, which consists of the constrained stochastic optimization problem for fair machine learning, the local differential privacy mechanism for releasing sensitive attributes, the learning procedure of fair model using private sensitive attributes, and the definition of asymptotic relative efficiency in terms of fairness violations. In Section 3, we develop theory for the privacy cost under a single exact fairness constraint and then generalize this theory to some extent. By simulating a risk-parity linear regression problem in Section 4, we validate our theory and illustrate the utility of our tools. Finally, we summarize our work in Section 5 and point out an interesting avenue of future work.

1.1. RELATED WORK

The interaction between fairness and privacy has been investigated from three perspectives: learning approximately fair models without sensitive attributes (Hashimoto et al., 2018; Lahoti et al., 2020) , learning approximately fair models with wildly noisy sensitive attributes (Kallus et al., 2019; Awasthi et al., 2020; Wang et al., 2020) , and learning exactly fair models with structured noisy sensitive attributes (Lamy et al., 2020; Mozannar et al., 2020) . This paper focuses on the third aspect. The works that are most pertinent to ours are Lamy et al. ( 2020 

2.1. FAIR MACHINE LEARNING AS CONSTRAINED STOCHASTIC OPTIMIZATION

In-processing fair machine learning is typically a supervised learning problem with fairness constraints (Zafar et al., 2017; Agarwal et al., 2018) . Such a problem can most often be formulated as a constrained stochastic optimization problem: (empirical) risk minimization subject to (empirical) fairness constraints. Demographic parity, also known as statistical parity, means that the prediction Y = f (X) is statistically independent of the protected attribute A. Definition 2.2 (Equality of opportunity). Let Y = 1 be the advantaged label that is associated with a positive outcome and Y = f (X) be the output of the classifier. Equality of opportunity entails P{ Y = 1 | A = a, Y = 1} = P{ Y = 1 | A = a ′ , Y = 1} for all a, a ′ ∈ A. Equality of opportunity, also known as true positive rate parity, means that the prediction Y = f (X) conditioned on the advantaged label Y = 1 is statistically independent of the protected attribute A. Given a parametric model space H = {f θ (•) : θ ∈ Θ} and loss function ℓ : Θ × X × Y → R + (where Θ ⊂ R d is a finite-dimensional parameter space), an in-processing fair ML routine is to minimize the (empirical) risk E ℓ(θ; X, Y ) while satisfying some fairness constraints. To keep things simple, we assume there are only two demographic groups; i.e. |A| = 2. Without loss of generality, we refer to one group as advantaged (A = 1) and the other as disadvantaged (A = 0). Consider fair learning with demographic parity as an example. At the population level, the goal is to solve the problem: θ ⋆ ∈ arg min θ∈Θ E ℓ(θ; X, Y ) subject to E 1{f θ (X) = 1}|A = 1 -E 1{f θ (X) = 1}|A = 0 = 0 , (2.1)



Consider a fair binary classification problem. Let X ⊂ R d be the input space, Y = {0, 1} be the set of possible labels, and A be the set of possible values of the protected/sensitive attribute. In this setup, training and test examples are tuples of the form (X, A, Y ) ∈ X × A × Y, and a classifier is a map f : X → {0, 1}. Two popular definitions of algorithmic fairness for binary classification are demographic parity (Dwork et al., 2011) and equality of opportunity (Hardt et al., 2016). Definition 2.1 (Demographic parity). Let Y = f (X) be the output of the classifier. Demographic parity entails P{ Y = 1 | A = a} = P{ Y = 1 | A = a ′ } for all a, a ′ ∈ A.

) andMozannar et al. (2020).Lamy  et al. (2020)  assume that the sensitive attributes are subject to noise from the mutually contaminated learning model. Under such a structured noise mechanism, the noise rates can be consistently estimated, and when enforcing fairness with regard to noisy groups, scaling the fairness tolerance parameter more tightly is all that is required. Mozannar et al. (2020) suggest a differentially private model to release the sensitive attributes, which is a special type of the mutually contaminated learning model. Under such a designed noise mechanism,Mozannar et al. (2020)  show that if the classifier is independent of the sensitive attributes, then exact fairness with regard to noisy sensitive attributes is equivalent to that with regard to true sensitive attributes. The idea of the equivalence can be found in Lamy et al. (2020) while Mozannar et al. (2020) put it into a formal statement. We basically study the statistical cost of privacy on the generalizability of fairness when using Lamy et al. (2020)'s method under Mozannar et al. (2020)'s privacy mechanism.

