CAFE: CATASTROPHIC DATA LEAKAGE IN FEDER-ATED LEARNING

Abstract

Private training data can be leaked through the gradient sharing mechanism deployed in machine learning systems, such as federated learning (FL). Increasing batch size is often viewed as a promising defense strategy against data leakage. In this paper, we revisit this defense premise and propose an advanced data leakage attack to efficiently recover batch data from the shared aggregated gradients. We name our proposed method as catastrophic data leakage in federated learning (CAFE). Comparing to existing data leakage attacks, CAFE demonstrates the ability to perform large-batch data leakage attack with high data recovery quality. Experimental results on vertical and horizontal FL settings have validated the effectiveness of CAFE in recovering private data from the shared aggregated gradients. Our results suggest that data participated in FL, especially the vertical case, have a high risk of being leaked from the training gradients. Our analysis implies unprecedented and practical data leakage risks in those learning settings.

1. INTRODUCTION

Federated learning (FL) (Chilimbi et al., 2014; Shokri & Shmatikov, 2015) is an emerging machine learning framework where a central server and multiple workers collaboratively train a machine learning model. Most of existing FL methods consider the setting where each worker has data of a different set of subjects but their data share many common features. This setting is also referred to data partitioned or horizontal FL (HFL). Unlike the HFL setting, in many learning scenarios, multiple workers handle data about the same set of subjects, but each has a different set of features. This case arises in financial and healthcare applications (Chen et al., 2020) . In these examples, data owners (e.g., financial institutions and hospitals) have different records of those users in their joint user base, so, by combining their features, they can establish a more accurate model. We refer to this setting as feature-partitioned or vertical FL (VFL). Compared with existing distributed learning paradigms, FL raises new challenges including the heterogeneity of data and the privacy of data (McMahan et al., 2017) . To protect data privacy, only model parameters and the change of parameters (e.g., gradients) are exchanged between server and workers (Li, 2014; Iandola et al., 2015) . Recent works have studied how a malicious worker can embed backdoors or replace the global model in FL (Bagdasaryan et al., 2018; Bhagoji et al., 2019; Xie et al., 2020) . As exchanging gradients is often viewed as privacy-preserving protocols, little attention has been paid to information leakage from public shared gradients and batch identities. In this context, inferring private user data from the gradients has received growing interests (Fredrikson et al., 2015; Hitaj et al., 2017; Melis et al., 2018) . A popular method that was termed deep leakage from gradients (DLG) has been developed in (Zhu et al., 2019) that infers training data in an efficient way without using any generative models or prior information. However, DLG lacks generalizability on model architecture and weight distribution initialization (Geiping et al., 2020) . In Zhao et al. ( 2020), an analytical approach has been developed to extract accurate labels from the gradients. Wang et al. (2020) proposed a novel gradient difference as a distance measure to improve recovery accuracy. However, all of them cannot scale up to the large-batch data leakage setting. The contributions of this paper are summarized in the following. 1) We develop an advanced data leakage attack that we term CAFE to overcome the limitation of current data leakage attacks on FL. CAFE is able to recover large-scale data both in VFL and HFL. 3) The effectiveness and practical risk induced from our data leakage algorithm is justified in the dynamic FL training setting when all parameters in the model are updated every iteration.

2. PRELIMINARY

FL can be categorized into horizontal and vertical FL settings (Kairouz et al., 2019) . In this section, we provide necessary background of FL in this section. Horizontal FL. In HFL, data are distributed among local workers holding the same feature space. Suppose that there are M workers participating in the FL process and the number of samples in the dataset X is N . The dataset is denoted as X := [X 1 , . . . , X m , . . . , X M ] T , where X m ∈ R Nm×p is the local data partitioned to worker m, and p is the dimension of data feature space, N m is the number of data samples partitioned to local worker m, and M m=1 N m = N . Since all local data share the same feature space, each local worker computes the gradients independently and uploads them to the server. The server receives all gradients from each local worker and uses gradient aggregation methods such as FedAvg (Konečnỳ et al., 2016) . Let the parameters of the model as θ and the loss function as L. Then the objective function of HFL can be defined as: min θ 1 N M m=1 L(θ; X m ) with L(θ; X m ) := n∈Nm L(θ; x n ) (1) Vertical FL. Different from HFL, in VFL, each local worker m is associated with a unique set of features. Each data sample x n in dataset X can be written as x n = [x T n1 , . . . , x T nm , . . . , x T nM ] T 2) where x nm ∈ R pm is the data partitioned to worker m and p m is the data dimension in local worker m. The label space {y n } N n=1 can be regarded as a special feature and is partitioned to the server or a certain local worker. Similar to (1), the objective function of VFL can be written as: (3)

3. CATASTROPHIC DATA LEAKAGE FROM BATCH GRADIENTS

To realize large-scale data recovery from aggregated gradients, we propose our algorithm named as CAFE: Catastrophic dAta leakage in Federated lEarning. While CAFE can be applied to any type of data, without loss of generality, we use image datasets throughout the paper.



Figure 1: Illustration of large-batch data leakage on CIFAR-10 from shared gradients in FL

x n1 ; . . . ; x nM )

