PREPARE FOR THE WORST: GENERALIZING ACROSS DOMAIN SHIFTS WITH ADVERSARIAL BATCH NOR-MALIZATION

Abstract

Adversarial training is the industry standard for producing models that are robust to small adversarial perturbations. However, machine learning practitioners need models that are robust to other kinds of changes that occur naturally, such as changes in the style or illumination of input images. Such changes in input distribution have been effectively modeled as shifts in the mean and variance of deep image features. We adapt adversarial training by adversarially perturbing these feature statistics, rather than image pixels, to produce models that are robust to distributional shifts. We also visualize images from adversarially crafted distributions. Our method, Adversarial Batch Normalization (AdvBN), significantly improves the performance of ResNet-50 on ImageNet-C (+8.1%), Stylized-ImageNet (+6.7%), and ImageNet-Instagram (+3.9%) over standard training practices. In addition, we demonstrate that AdvBN can also improve generalization on semantic segmentation.

1. INTRODUCTION

Robust optimization for neural networks has been a major focus of recent research. A mainstream approach to reducing the brittleness of classifiers is adversarial training, which solves a min-max optimization problem in which an adversary makes perturbations to images to degrade network performance, while the network adapts its parameters to resist degradation (Goodfellow et al., 2015; Kurakin et al., 2017; Madry et al., 2018) . The result is a hardened network that is no longer brittle to small perturbations to input pixels. While adversarial training makes networks robust to adversarial perturbations, it does not address other forms of brittleness that plague vision systems. For example, shifts in image style, lighting, color mapping, and domain shifts can still severely degrade the performance of neural networks (Hendrycks & Dietterich, 2019) . We propose adapting adversarial training to make neural networks robust to changes in image style and appearance, rather than small perturbations at the pixel level. We formulate a min-max game in which an adversary chooses adversarial feature statistics, and network parameters are then updated to resist these changes in feature space that correspond to appearance differences of input images. This



Figure 1: Images from ImageNet variants along with classification scores by a pre-trained ResNet-50 model. The right-most image is generated by our Adversarial Batch Normalization module. Details on how we generate this image can be found in Section 3.



