EFFICIENT ONLINE AUGMENTA-TION WITH RANGE LEARNING

Abstract

State-of-the-art automatic augmentation methods (e.g., AutoAugment and Ran-dAugment) for visual recognition tasks diversify training data using a large set of augmentation operations. The range of magnitudes of many augmentation operations (e.g., brightness and contrast) is continuous. Therefore, to make search computationally tractable, these methods use fixed and manually-defined magnitude ranges for each operation, which may lead to sub-optimal policies. To answer the open question on the importance of magnitude ranges for each augmentation operation, we introduce RangeAugment that allows us to efficiently learn the range of magnitudes for individual as well as composite augmentation operations. RangeAugment uses an auxiliary loss based on image similarity as a measure to control the range of magnitudes of augmentation operations. As a result, RangeAugment has a single scalar parameter for search, image similarity, which we simply optimize via linear search. RangeAugment integrates seamlessly with any model and learns model-and task-specific augmentation policies. With extensive experiments on the ImageNet dataset across different networks, we show that RangeAugment achieves competitive performance to state-of-the-art automatic augmentation methods with 4-5 times fewer augmentation operations. Experimental results on semantic segmentation and contrastive learning further shows RangeAugment's effectiveness.

1. INTRODUCTION

Data augmentation is a widely used regularization method for training deep neural networks (LeCun et al., 1998; Krizhevsky et al., 2012; Szegedy et al., 2015; Perez & Wang, 2017; Steiner et al., 2021) . These methods apply carefully designed augmentation (or image transformation) operations (e.g., color transforms) to increase the quantity and diversity of training data, which in turn helps improve the generalization ability of models. However, these methods rely heavily on expert knowledge and extensive trial-and-error experiments. Recently, automatic augmentation methods have gained attention because of their ability to search for augmentation policy (e.g., combinations of different augmentation operations) that maximizes validation performance (Cubuk et al., 2019; 2020; Lim et al., 2019; Hataya et al., 2020; Zheng et al., 2021) . In general, most augmentation operations (e.g., brightness and contrast) have two parameters: (1) the probability of applying them and (2) their range of magnitudes. These methods take a set of augmentation operations with a fixed (often discretized) range of magnitudes as an input, and produce a policy of applying some or all augmentation operations along with their parameters (Fig. 1 ). As an example, AutoAugment (Cubuk et al., 2019) discretizes the range of magnitudes and probabilities of 16 augmentation operations, and searches for sub-policies (i.e., composition of two augmentation operations along with their probability and magnitude) in a space of about 10 32 possible combinations. These methods empirically show that automatic augmentation policies help improve performance of downstream networks. For example, AutoAugment improves the validation top-1 accuracy of ResNet-50 (He et al., 2016) by about 1.3% on the ImageNet dataset (Deng et al., 2009) . In other words, these methods underline the importance of automatic composition of augmentation operations in improving validation performance. However, policies generated using these networks may be sub-optimal because they use hand-crafted magnitude ranges. The importance of magnitude ranges for each augmentation operation is still an open question. An obstacle in

Search space

Set of augmentations T with their parameters

• Probability of applying augmentation operation

• Augmentation magnitude, etc. Search strategy answering this question is the range of magnitudes for most augmentation operations is continuous, which makes the search computationally intractable. This paper introduces RangeAugment, a simple and efficient method to learn the range of magnitudes for each augmentation operation. Inspired by image similarity metrics (Hore & Ziou, 2010), RangeAugment introduces an auxiliary augmentation loss that allows us to learn the range of magnitudes for each augmentation operation. We realize this by controlling the similarity between the input and the augmented image for a given model and task. Rather than directly specifying the parameters for each augmentation operation, RangeAugment takes a target image similarity value as an input. The loss function is then formulated as a combination of the empirical loss and an augmentation loss. The objective of the augmentation loss is to match the target image similarity value. Therefore, the search objective in RangeAugment is to find the target similarity value that provides a good trade-off between minimizing the augmentation loss (i.e., matching the target similarity value) and the empirical loss. As a result, the augmentation policy learning in RangeAugment reduces to searching for a single scalar parameter, target image similarity, that maximizes downstream model's validation performance. We search for this target image similarity value via linear search. Empirically, we observe that this trade-off between the augmentation and empirical loss leads to better generalization ability of downstream model. Compared to existing automatic augmentation methods that require a large set of augmentation operations (usually 14-16 operations), RangeAugment is able to achieve competitive performance with only three simple operations (brightness, contrast, and additive Gaussian noise). Because RangeAugment's search space is independent of augmentation parameters and is fully differentiable (Fig. 1 ), it can be trained end-to-end with any downstream model to learn model-and task-specific policies (Fig. 2 ). We empirically demonstrate in Section 4 that RangeAugment allows us to learn model-specific policies when trained end-to-end with downstream models on the ImageNet dataset (Fig. 2a ). Im-



Figure 1: Comparison between RangeAugment and standard automatic augmentation methods.RangeAugment's search space is independent of augmentation parameters, allowing us to learn model-and task-specific policies in a constant time.

