FAIR MIXUP: FAIRNESS VIA INTERPOLATION

Abstract

Training classifiers under fairness constraints such as group fairness, regularizes the disparities of predictions between the groups. Nevertheless, even though the constraints are satisfied during training, they might not generalize at evaluation time. To improve the generalizability of fair classifiers, we propose fair mixup, a new data augmentation strategy for imposing the fairness constraint. In particular, we show that fairness can be achieved by regularizing the models on paths of interpolated samples between the groups. We use mixup, a powerful data augmentation strategy to generate these interpolates. We analyze fair mixup and empirically show that it ensures a better generalization for both accuracy and fairness measurement in tabular, vision, and language benchmarks. The code is available at https://github.com/chingyaoc/fair-mixup.

1. INTRODUCTION

Fairness has increasingly received attention in machine learning, with the aim of mitigating unjustified bias in learned models. Various statistical metrics were proposed to measure the disparities of model outputs and performance when conditioned on sensitive attributes such as gender or race. Equipped with these metrics, one can formulate constrained optimization problems to impose fairness as a constraint. Nevertheless, these constraints do not necessarily generalize since they are data-dependent, i.e they are estimated from finite samples. In particular, models that minimize the disparities on training sets do not necessarily achieve the same fairness metric on testing sets (Cotter et al., 2019) . Conventionally, regularization is required to improve the generalization ability of a model (Zhang et al., 2016) . On one hand, explicit regularization such as weight decay and dropout constrain the model capacity. On the other hand, implicit regularization such as data augmentation enlarge the support of the training distribution via prior knowledge (Hernández-García & König, 2018) . In this work, we propose a data augmentation strategy for optimizing group fairness constraints such as demographic parity (DP) and equalized odds (EO) (Barocas et al., 2019) . Given two sensitive groups such as male and female, instead of directly restricting the disparity, we propose to regularize the model on interpolated distributions between them. Those augmented distributions form a path connecting the two sensitive groups. Figure 1 provides an illustrative example of the idea. The path simulates how the distribution transitions from one group to another via interpolation. Ideally, if the model is invariant to the sensitive attribute, the expected prediction of the model along the path should have a smooth behavior. Therefore, we propose a regularization that favors smooth transitions along the path, which provides a stronger prior on the model class. We adopt mixup (Zhang et al., 2018b) , a powerful data augmentation strategy, to construct the interpolated samples. Owing to mixup's simple form, the smoothness regularization we introduce has a closed form expression that can be easily optimized. One disadvantage of mixup is that the interpolated samples might not lie on the natural data manifold. Verma et al. (2019) propose Manifold Mixup, which generate the mixup samples in a latent space. Previous works (Bojanowski et al., 2018; Berthelot et al., 2018) have shown that interpolations between a pair of latent features correspond to semantically meaningful, smooth interpolation in the input space. By constructing the path in the latent space, we can better capture the semantic changes while traveling between the sensitive groups and hence result in a better fairness regularizer that we coin fair mixup. Empirically, fair mixup improves the generalizability for both DP and EO on tabular, computer-vision, and natural language benchmarks. Theoretically, we prove for a particular case that fair mixup corresponds to a Mahalanobis metric in the feature space in which we perform the classification. This metric ensures group fairness of the model, and involves the Jacobian of the feature map as we travel along the path. In short, this work makes the following contributions: • We develop fair mixup, a data augmentation strategy that improves the generalization of group fairness metrics; • We provide a theoretical analysis to deepen our understanding of the proposed method; • We evaluate our approach via experiments on tabular, vision, and language benchmarks;

2. RELATED WORK

Machine Learning Fairness To mitigate unjustified bias in machine learning systems, various fairness definitions have been proposed. The definitions can usually be classified into individual fairness or group fairness. A system that is individually fair will treat similar users similarly, where the similarity between individuals can be obtained via prior knowledge or metric learning (Dwork et al., 2012; Yurochkin et al., 2019) . Group fairness metrics measure the statistical parity between subgroups defined by the sensitive attributes such as gender or race (Zemel et al., 2013; Louizos et al., 2015; Hardt et al., 2016) . While fairness can be achieved via pre-or post-processing, optimizing fair metrics at training time can lead to the highest utility (Barocas et al., 2019) 2019) inherit the two-player setting while training each player on two separated datasets. In spite of the analytical solutions and theoretical guarantees, game-theoretic approaches could be hard to scale for complex model classes. In contrast, our proposed fair mixup, is a general data augmentation strategy for optimizing the fairness constraints, which is easily compatible with any dataset modality or model class. Data Augmentation and Regularization Data augmentation expands the training data with examples generated via prior knowledge, which can be seen as an implicit regularization (Zhang et al., 2016; Hernández-García & König, 2018) where the prior is specified as virtual examples. Zhang et al. (2018b) proposes mixup, which generate augmented samples via convex combinations of pairs of examples. In particular, given two examples z i , z j ∈ R d where z could include both input and label, mixup constructs virtual samples as tz i + (1 -t)z j for t ∈ [0, 1]. State-of-the-art results



Figure 1: (a) Visualization of the path constructed via mixup interpolations between groups that have distribution P 0 and P 1 , respectively. (b) Fair mixup penalizes the changes in model's expected prediction with respect to the interpolated distributions. The regularized model (blue curve) has smaller slopes comparing to the unregularized one (orange curve) along the path from P 0 to P 1 , which eventually leads to smaller demographic parity ∆DP.

. For instance, Woodworth et al. (2017) impose independence via regularizing the covariance between predictions and sensitive attributes. Zafar et al. (2017) regularize decision boundaries of convex margin-based classifier to minimize the disparaty between groups. Zhang et al. (2018a) mitigate the bias via minimizing an adversary's ability to predict sensitive attributes from predictions. Nevertheless, these constraints are data-dependent, even though the constraints are satisfied during training, the model may behave differently at evaluation time. Agarwal et al. (2018) analyze the generalization error of fair classifiers obtained via two-player games. To improve the generalizability, Cotter et al. (

