THE COST OF PRIVACY IN FAIR MACHINE LEARNING Anonymous authors Paper under double-blind review

Abstract

A common task in fair machine learning is training ML models that preserve certain summary statistics across subpopulations defined by sensitive attributes. However, access to such sensitive attributes in training data is restricted and the learner must rely on noisy proxies for the sensitive attributes. In this paper, we study the effect of a privacy mechanism that obfuscates the sensitive attributes from the learner on the fairness of the resulting classifier. We show that the cost of privacy in fair ML is a decline in the generalizability of fairness constraints.

1. INTRODUCTION

The fairness of machine learning systems is gaining increasing attention in recent years. Among the numerous fairness objectives is ensuring that a machine learning model does not discriminate against subpopulations that are typically identified by sensitive attributes (e.g., race, gender). When training a fair model and evaluating model bias, it is necessary to possess sensitive attributes; however, access to and use of such sensitive data is frequently prohibited by laws and regulations. Credit card companies, for instance, are not permitted to inquire about a person's race when they apply for credit, yet they must demonstrate that their decisions are not discriminatory (Chen et al., 2019) . Ideally, sensitive personal information should not be disclosed during the training of ML models. However, it is impossible to ensure exact notions of fairness (such as demographic parity or equality of opportunity) without any knowledge of the sensitive data. Fortunately, differential privacy (Dwork et al., 2006 ) is a promising workaround, which can offer a graceful compromise between privacy and utility. Mozannar et al. (2020) propose to release sensitive attributes in a locally differentially private way: adding noise to the sensitive data so that adversaries cannot infer any information with high confidence about a single record. The advantage of the privacy mechanism proposed by Mozannar et al. ( 2020) is an invariance property: exact notions of fairness with regard to true sensitive attributes and noisy sensitive attributes are equivalent. An implication of the invariance property is that the optimal model of fairness can be learned at the population level. Nonetheless, it remains unclear what the precise statistical impact of privacy on fairness is. In this work, we study the statistical cost of privacy on fairness in the task of learning fair ML models with differentially private sensitive attributes. The main benefits of the developed theory are 1. statistically principled: We propose a statistically principled metric to characterize the cost of privacy on fairness. A restricted notion of statistical efficiency precisely quantifies the privacy cost asymptotically. 2. interpretable: Privacy leads to a decline in the statistical efficiency. Such efficiency loss is interpretable: it explicitly depends on the privacy budget, the subpopulation imbalance level, and few other problem-specific parameters. The rest of this paper is organized as follows. In Section 2, we formalize the problem setup, which consists of the constrained stochastic optimization problem for fair machine learning, the local differential privacy mechanism for releasing sensitive attributes, the learning procedure of fair model using private sensitive attributes, and the definition of asymptotic relative efficiency in terms of fairness violations. In Section 3, we develop theory for the privacy cost under a single exact fairness constraint and then generalize this theory to some extent. By simulating a risk-parity linear regression problem in Section 4, we validate our theory and illustrate the utility of our tools. Finally, we summarize our work in Section 5 and point out an interesting avenue of future work.

