ON FAST ADVERSARIAL ROBUSTNESS ADAPTATION IN MODEL-AGNOSTIC META-LEARNING

Abstract

Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. It enables us to learn a meta-initialization of model parameters (that we call meta-model) to rapidly adapt to new tasks using a small amount of labeled training data. Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning. In addition to generalization, robustness is also desired for a meta-model to defend adversarial examples (attacks). Toward promoting adversarial robustness in MAML, we first study when a robustness-promoting regularization should be incorporated, given the fact that MAML adopts a bi-level (fine-tuning vs. meta-update) learning procedure. We show that robustifying the meta-update stage is sufficient to make robustness adapted to the task-specific fine-tuning stage even if the latter uses a standard training protocol. We also make additional justification on the acquired robustness adaptation by peering into the interpretability of neurons' activation maps. Furthermore, we investigate how robust regularization can efficiently be designed in MAML. We propose a general but easily-optimized robustness-regularized meta-learning framework, which allows the use of unlabeled data augmentation, fast adversarial attack generation, and computationally-light fine-tuning. In particular, we for the first time show that the auxiliary contrastive learning task can enhance the adversarial robustness of MAML. Finally, extensive experiments are conducted to demonstrate the effectiveness of our proposed methods in robust few-shot learning. Codes are available at https://github.com/wangren09/MetaAdv.

1. INTRODUCTION

Meta-learning, which can offer fast generalization adaptation to unseen tasks (Thrun & Pratt, 2012; Novak & Gowin, 1984) , has widely been studied from model-and metric-based methods (Santoro et al., 2016; Munkhdalai & Yu, 2017; Koch et al., 2015; Snell et al., 2017) to optimizationbased methods (Ravi & Larochelle, 2016; Finn et al., 2017; Nichol et al., 2018) . In particular, model-agnostic meta-learning (MAML) (Finn et al., 2017) is one of the most intriguing bi-level optimization-based meta-learning methods designed for fast-adapted few-shot learning. That is, the learnt meta-model can rapidly be generalized to unforeseen tasks with only a small amount of data. It has successfully been applied to use cases such as object detection (Wang et al., 2020) , medical image analysis (Maicas et al., 2018) , and language modeling (Huang et al., 2018) . In addition to generalization-ability, recent works (Yin et al., 2018; Goldblum et al., 2019; Xu et al., 2020) investigated MAML from another fundamental perspective, adversarial robustness, given by the capabilities of a model defending against adversarially perturbed inputs (known as adversarial

