A SIMULATION-BASED FRAMEWORK FOR ROBUST FEDERATED LEARNING TO TRAINING-TIME ATTACKS Anonymous

Abstract

Well-known robust aggregation schemes in federated learning (FL) are shown to be vulnerable to an informed adversary who can tailor training-time attacks (Fang et al., 2020; Xie et al., 2020). We frame robust distributed learning problem as a game between a server and an adversary that is able to optimize strong trainingtime attacks. We introduce RobustTailor, a simulation-based framework that prevents the adversary from being omniscient. The simulated game we propose enjoys theoretical guarantees through a regret analysis. RobustTailor improves robustness to training-time attacks significantly while preserving almost the same privacy guarantees as standard robust aggregation schemes in FL. Empirical results under challenging attacks show that RobustTailor performs similar to an upper bound with perfect knowledge of honest clients.

1. INTRODUCTION

In federated learning (FL), a global/personalized model is learnt from data distributed on multiple clients without sharing data (McMahan et al., 2017; Kairouz et al., 2021) . Clients compute their (stochastic) gradients using their own local data and send them to a central server for aggregating and updating a model. While FL offers improvements in terms of privacy, it creates additional challenges in terms of robustness. Clients are often prone to the bias in the stochastic gradient updates, which comes not only from poor sampling or data noise but also from malicious attacks of Byzantine clients who may send arbitrary messages to the server instead of correct gradients (Guerraoui et al., 2018) . Therefore, in FL, it is essential to guarantee some level of robustness to Byzantine clients that might be compromised by an adversary. Compromised clients are vulnerable to data/model poisoning and tailored attacks (Fang et al., 2020) . Byzantine-resilience is typically achieved by robust gradient aggregation schemes e.g., Krum (Blanchard et al., 2017 ), Comed (Yin et al., 2018 ), and trimmedmean (Yin et al., 2018) . These aggregators are resilient against attacks that are designed in advance. However, such robustness is insufficient in practice since a powerful adversary could learn the aggregation rule and tailor its training-time attack. It has been shown that well-known Byzantine-resilient gradient aggregation schemes are susceptible to an informed adversary that can tailor the attacks (Fang et al., 2020) . Specifically, Fang et al. (2020) and Xie et al. (2020) proposed efficient and nearly optimal trainingtime attacks that circumvent Krum, Comed, and trimmedmean. A tailored attack is designed with a prior knowledge of the robust aggregation rule used by the server, such that the attacker has a provable way to corrupt the training process. Given the information leverage of the adversary, it is a significant challenge to establish successful defense mechanisms against such tailored attacks. In this paper, we formulate robust distributed learning problem against training-time attacks as a game between a server and an adversary. To prevent the adversary from being omniscient, we propose to follow a mixed strategy using the existing robust aggregation rules. In real-world settings, both server and adversary have a number of aggregation rules and attack programs. How to utilize these aggregators efficiently and guarantee robustness is a challenging task. We address scenarios where neither the specific attack method is known in advance by the aggregator nor the exact aggregation rule used in each iteration is known in advance by the adversary, while the adversary and the server know the set of server's aggregation rules and the set of attack programs, respectively. 1 Due to information asymmetry between the server and the adversary, we assume every client donates a small amount of honest data to the server as the price to achieve some level of security more proactively and efficiently. Providing such public dataset to achieve robustness is a common assumption in FL (Fang & Ye, 2022; Huang et al., 2022; Kairouz et al., 2021; Yoshida et al., 2020; Zhao et al., 2018; Fang et al., 2020; Xie et al., 2020; Cao & Lai, 2019; Chang et al., 2019; Cao et al., 2020) . We propose RobustTailor, a scheme based on simulating aggregation rules under different attacks. With minimal privacy leakage, RobustTailor realizes high resilience to training-time attacks. RobustTailor maintains stable performance under a challenging mixed attack, a strategy we propose for the adversary to simulate and design a successful attack when a smart server uses a mixed strategy to make the problem of attack design computationally harder. We emphasize that any deterministic Byzantine-resilient algorithm can be added in server's aggregation pool. Similarly, any attack can be used in the set of adversary's attack programs.

1.1. SUMMARY OF CONTRIBUTIONS

• We frame robust distributed learning problem as a game between a server and an adversary that tailors training-time attacks. • We propose a simulation-based framework RobustTailor to improve robustness by preventing the adversary from being omniscient. • The simulated game we propose enjoys theoretical guarantees through a regret analysis. • Empirical studies validate our theory and show that RobustTailor preforms similar to an upper bound with perfect knowledge of all honest clients over the course of training. Even under a challenging mixed attack strategy, RobustTailor outperforms the robust baselines in terms of robustness and accuracy.

1.2. RELATED WORK

In this section, we provide a summary of related work. See Appendix A for complete related work. Training-time attacks in FL. Federated learning (FL) usually suffers from training-time attacks (Biggio et al., 2012; Bhagoji et al., 2019; Sun et al., 2019; Bagdasaryan et al., 2020) because the server trains the model across various unreliable clients with private datasets. A strong adversary can potentially participate in every training round and adapt its attacks to an updated model. In model update poisoning, a class of training-time attacks, an adversary controls some clients and directly manipulates their outputs aiming to bias the global model towards opposite direction of honest training (Kairouz et al., 2021) . If Byzantine clients have access to the updates of honest clients, they can tailor their attacks and make them difficult to detect (Fang et al., 2020; Xie et al., 2020) . Robust aggregation and Byzantine resilience. To improve robustness under general Byzantine clients, a number of robust aggregation schemes have been proposed, which are mainly inspired by robust statistics such as median-based aggregators (Yin et al., 2018; Chen et al., 2017) 



While this assumption is essential to frame our game, we provide experimental results on challenging settings where the server does not know the set of attack programs in Section 5.



, Krum (Blanchard et al., 2017), trimmed mean (Yin et al., 2018). Moreover, Fang et al. (2020); Xie et al. (2020); Cao & Lai (2019); Cao et al. (2020) propose server-side verification methods using auxiliary data. Karimireddy et al. (2021) and Alistarh et al. (2018) propose history-aided aggregators. Ramezani-Kebrya et al. (2022) propose a framework based on randomization of multiple aggregation rules. However, none of them selects a proper aggregation rule proactively during training as our framework RobustTailor, and all of them can be used in RobustTailor while we mainly focus on statisticalbased aggregators in this paper. Although past work has shown that these aggregators can defend successfully under specific conditions (Blanchard et al., 2017; Chen et al., 2017; Su & Vaidya, 2016), Fang et al. (2020) and Xie et al. (2020) argue that Byzantine-resilient aggregators can fail when an informed adversary tailor a careful attack and Gouissem et al. (2022) proves that such aggregation rules are vulnerable. Therefore, developing a robust and efficient algorithm under such strong tailored attacks is essential to improve security of FL, which is the goal of this paper.Game theory in FL. Online convex optimization (OCO) framework(Zinkevich, 2003)  is widely influential in the learning community(Hazan et al., 2016; Shalev-Shwartz et al., 2012), and bandit convex optimization (BCO) as an extension of OCO was proposed byAwerbuch & Kleinberg (2008)   for decision making with limited feedback. Bandit paradigms paralleling FL framework are proposed by Shi & Shen (2021) and its extension under Byzantine attacks is proposed byDemirel et al.  (2022). However, they account for uncertainties from both arm and client sampling rather than robust

