TACKLING IMBALANCED CLASS IN FEDERATED LEARNING VIA CLASS DISTRIBUTION ESTIMATION Anonymous

Abstract

Federated Learning (FL) has become an upsurging machine learning method due to its applicability in large-scale distributed system and its privacy-preserving property. However, in real-world applications, the presence of class imbalance issue, especially the mismatch between local and global class distribution, greatly degrades the performance of FL. Moreover, due to the privacy constrain, the class distribution information of clients can not be accessed directly. To tackle class imbalance issue under FL setting, a novel algorithm, FedRE, is proposed in this paper. We propose a new class distribution estimation method for the FedRE algorithm, which requires no extra client data information and thus has no privacy concern. Both experimental results and theoretical analysis are provided to support the validity of our distribution estimation method. The proposed algorithm is verified with several experiment, including different datasets with the presence of class imbalance and local-global distribution mismatch. The experimental results show that FedRE is effective and it outperforms other related methods in terms of both overall and minority class classification accuracy.

1. INTRODUCTION

Federated Learning (FL) was first proposed (McMahan et al., 2017) when they were developing the application of next-word prediction on mobile keyboard. It enables multiple clients to collaboratively learn a machine learning model, without sharing their locally stored raw data (Li et al., 2020a) . This property greatly reduce the communication cost and preserve the privacy of clients, which makes FL become an upsurging research direction, not only in machine learning community but also in a variety of engineering applications, including communication (Wang et al., 2022; Niknam et al., 2020; Mills et al., 2019 ), edge computing (Zhang et al., 2021a; Wang et al., 2019a; b) , and energy engineering (Saputra et al., 2019; Hamdi et al., 2021; Cheng et al., 2022) . Standard FL consists of four major steps, which are client selection, broadcast, client computation, and aggregation (Kairouz et al., 2021) . In each iteration, the central server will select a subset of clients in each global iteration at first, and then broadcast the global model to them. After receiving the global model, the selected clients will perform model update based on their local dataset given the global model as initial condition, and then upload updates to the server. As the final step, the server will aggregate the collected information to update the global model, and then start a new iteration. In FL framework, one of the most difficult challenges is class imbalance issue. Class imbalance means that the data distribution among all classes are not uniform. In other words, the majority of data samples may belong to certain classes, and other minority classes may only have a small amount of data. Class imbalance results in low classification accuracy on minority classes, and also slow down the training speed. In the literature, several methods have been proposed to resolve class imbalance issue in centralized machine learning scheme. In general, the methods can be categorized as data-level methods, algorithm level methods, and hybrid methods (Johnson & Khoshgoftaar, 2019) . For data-level method, Jo & Japkowicz (2004) proposed a cluster-based sampling scheme to tackle the class imbalance issue. For algorithm level method, Ling & Sheng (2008) proposed the cost-sensitive learning to improve the classification performance on minority class. For hybrid method, Sun et al. ( 2007) integrated both sampling techniques and cost-sensitive learning and showed a significant performance boost in most of the cases. However, in FL, since all training data

