LONG TERM FAIRNESS VIA PERFORMATIVE DISTRI-BUTIONALLY ROBUST OPTIMIZATION

Abstract

Fairness researchers in machine learning (ML) have coalesced around several fairness criteria which provide formal definitions of what it means for an ML model to be fair. However, these criteria have some serious limitations. We identify four key shortcomings of these formal fairness criteria and address them by extending performative prediction to include a distributionally robust objective. Performative prediction is a recent framework developed to understand the effects of when deploying a model influences the distribution on which it is making predictions. We prove a convergence result for our proposed repeated distributionally robust optimization (RDRO). We further verify our results empirically and develop experiments to demonstrate the impact of using RDRO on learning fair ML models.

1. INTRODUCTION

In the past two decades, machine learning (ML) has moved from the confines of research institutes and university laboratories to become a core element of the global economy. ML models are now deployed at enormous scales in complex environments, often making high stakes decision. Too often, however, this is done without adequate concern for the fairness and robustness of these ML models. Fairness in ML is a burgeoning research area, but much of the work in fairness, particularly in defining formal fairness criteria, has been limited to the static classification setting. Woodworth et al. (2017) . These formal fairness criteria assume a sensitive characteristic or protected demographic group for whom we want to ensure our model is non-discriminatory. The fairness criteria are then properties of the joint distribution of this characteristic, the output of the classifier, and the true labels of the data.

Efforts

While these fairness definitions have been a useful starting point in the consideration of discrimination by ML models, they have several limitations. 1. They are not equivalent and, in most scenarios, they are incompatible. 2. They apply only to static supervised learning problems and ignore the dynamic environments characteristic of many real world scenarios with fairness concerns. 3. They rely on having access to demographic information. The definitions can only be used if one has access to the sensitive characteristic, which is often not the case. 4. They ignore intersectionality. The criteria do not take into account individuals who may sit at the intersection of several sensitive demographic groups. Fairness is fundamentally a philosophical and political question, and the notion of having a single, universal formal definition of fairness for ML is likely naïve. For this reason, this work does not attempt to formally define fairness and largely ignores the first problem noted above. We do, however, attempt to address issues 2, 3, and 4 by drawing upon two recent areas of research with implications for fairness in ML: performative prediction and distributionally robust optimization (DRO). DRO offers a compelling and flexible method for training non-discriminatory algorithms without needing access to demographic information. Performative prediction, on the other hand, attempts to outline a theoretical framework through which we can reason about ML models in dynamic environments, when the act of deploying a model influences the distribution on which it is making decisions. We combine these two areas of research to extend the performative prediction framework developed in Perdomo et al. (2020) . The work in performative prediction has thus far only concerned risk minimization and empirical risk minimization (ERM), so we extend definitions to include a distributionally robust objective and prove an analogous convergence result to that shown in Perdomo et al. (2020) . Due to space constraints, we provide an extended discussion of related work in section A.1 of the appendix.

2.1. DISTRIBUTIONALLY ROBUST OPTIMIZATION

The de facto objective used in most supervised learning settings is ERM. In ERM we attempt to approximate minimization of the expected loss over the data generating distribution by minimizing the average loss over our data set: ĥ = arg min h∈H 1 n n i=1 ℓ(h(x i ), y i ), where h represents a hypothesis or model, H a hypothesis class, and ℓ a non-negative loss function. ERM is intuitively appealing and has important theoretical guarantees associated with it Vapnik (1991) . It can, however, be problematic when it comes to fairness concerns. Since we are averaging the loss over our data points, in general, ERM causes an algorithm to focus on majority cases while ignoring minority cases or rare events. DRO, on the other hand, considers the distributionally robust problem in which we construct an uncertainty set around the data generating distribution and attempt to minimize the expected loss on the worst-case distribution within this uncertainty set. Following Duchi & Namkoong (2021) we define our uncertainty set as U f (P ) = {Q : D f (Q||P ) ≤ ρ}, where D f (Q||P ) is an f -divergence between probability distributions Q and P . Formally the distributionally robust problem is as follows: minimize θ∈Θ sup Q≪P0 {E Q [ℓ(θ; X)] : Q ∈ U f (P 0 )} , where Θ ⊂ R d is the parameter (model) space, P 0 is the data generating distribution on the measure space (X , A), X is a random element of X and ℓ : Θ × X → R is a loss function. In this formulation of DRO, the uncertainty set is determined by an f -divergence between Q and P 0 and {E Q [ℓ(θ; X)] : Q ∈ U f (P 0 )} is the set of all expected losses over the f -divergence ball of radius ρ, centred at P 0 . Alternative DRO formulations utilizing different measures of distance between probability distributions such as Wasserstein balls have also been explored Wald (1945) ; Wozabal ( 2012 2018). Note that the distributionally robust objective does not require any access to demographic information and, at least potentially, naturally accounts for intersectionality. One can optimize the distributionally robust objective directly via the primal form, specified above, or alternatively through a dual formulation. For convex losses, the dual form is jointly convex in the parameters of the model and the dual variables Duchi & Namkoong (2021) . A detailed discussion of the dual formulation can be found in A.2. The dual formulation also helps provide intuition for why DRO is more likely to result in fair ML models.

2.2. PERFORMATIVE PREDICTION

Performative prediction attempts to formalize the notion of a model affecting the distribution on which it is making predictions in a type of feedback loop. There are many examples of scenarios with



to define fairness in ML have resulted in myriad criteria being proposed, many of which are equivalent to, or relaxations of, three core definitions of fairness: independence, separation, and sufficiency Barocas et al. (2019); Chouldechova (2017); Corbett-Davies et al. (2017); Dwork et al. (2012); Hardt et al. (2016b); Berk et al. (2021); Zafar et al. (2017); Kleinberg et al. (2017);

); Pflug & Wozabal (2007); Lee & Raginsky (2018). Unlike ERM, DRO does not equally weight each data point, but instead up-weights data points on which the model is achieving high loss. This means that the model should achieve somewhat uniform performance on individuals across demographic groups Duchi & Namkoong (2021); Duchi et al. (2020); Namkoong & Duchi (2016); Hashimoto et al. (

