SMOOTHED-SGDMAX: A STABILITY-INSPIRED AL-GORITHM TO IMPROVE ADVERSARIAL GENERALIZA-TION

Abstract

Unlike standard training, deep neural networks can suffer from serious overfitting problems in adversarial settings. Recent research (Xing et al., 2021b; Xiao et al., 2022) suggested that adversarial training can have nonvanishing generalization error even if the sample size n goes to infinity. A natural question arises: can we eliminate the generalization error floor in adversarial training? This paper gives an affirmative answer. First, by an adaptation of information-theoretical lower bound on the complexity of solving Lipschitz-convex problems using randomized algorithms, we establish a minimax lower bound Ω(s(T )/n) given a training loss of 1/s(T ) for the generalization gap in non-smooth settings, where T is the number of iterations, and s(T ) → +∞ as T → +∞. Next, by observing that the nonvanishing generalization error of existing adversarial training algorithms comes from the non-smoothness of the adversarial loss function, we employ a smoothing technique to smooth the adversarial loss function. Based on the smoothed loss function, we prove that a smoothed version of SGDmax algorithm can achieve a generalization bound O(s(T )/n), which eliminates the generalization error floor and matches the minimax lower bound. Experimentally, we show that the Smoothed-SGDmax algorithm improves adversarial generalization on common datasets.

1. INTRODUCTION

Deep neural networks (DNNs) (Krizhevsky et al., 2012; Hochreiter & Schmidhuber, 1997 ) is successful and rarely suffered overfitting issues (Zhang et al., 2021) . This phenomenon is also called benign overfitting. A well-trained neural network model can generalize well to the test data. However, in adversarial machine learning, overfitting becomes a serious issue (Rice et al., 2020) . Before the training algorithms converge, the robust test error starts to increase. This special type of overfitting is called robust overfitting and can be observed in the experiments on common datasets. See Fig. 1 , orange curve. Therefore, mitigating the robust overfitting is important to increase the adversarial robustness of a DNN model. Several recent works tried to figure out the causes of robust overfitting and designed methods to mitigate it. See the discussion in Sec. 2. A recent line of work (Xing et al., 2021b; Xiao et al., 2022) studied the robust overfitting issue of adversarial training from a theoretical perspective, using the notion of uniform algorithmic stability. Uniform algorithmic stability (UAS) (Bousquet & Elisseeff, 2002) was introduced to bound the generalization gap in machine learning problems. It provides algorithm-specific generalization bounds instead of algorithm-free generalization bounds such as classical results on VC-dimension (Vapnik & Chervonenkis, 2015) and Rademacher complexity (Bartlett & Mendelson, 2002) . Such stability-based generalization bounds provide insight into understanding the generalization ability of neural network models trained by different algorithms. Traditional adversarial training is to perform stochastic gradient descent (SGD) on the max function of the standard counterpart, which is also called SGDmax (Farnia & Ozdaglar, 2021 ). We will not distinguish two algorithms, "SGDmax" and "adversarial training (AT)", in the paper. The work of (Xing et al., 2021b; Xiao et al., 2022) both showed that SGDmax incurs a stability-based generalization bound in O(c(T ) + s(T )/n). Here T is the number of iterations, n is the number of samples, s(T ) is a function satisfies s(T ) → +∞ as T → +∞, and c(T ) is a sample size-independent  SGDmax O(c 1 (T ) + s(T ) n ) Ω(c 2 (T ) + s(T ) n ) Smoothed-SGDmax O( s(T ) n ) Ω( s(T ) n ) term and increase with T . Details of the form of s(T ), c(T ) are discussed in Sec. 4 and Sec. 5. They also provided the matching lower bounds to show that the sample size-independent term is unavoidable for SGDmax-based adversarial training algorithms. It provides a possible explanation of robust overfitting: even though we have arbitrarily large number of training samples, the adversarial generalization gap still does not vanish. The first question arises: what is the lower bound of the generalization gap for algorithms in adversarial machine learning settings? To answer this question, we develop a minimax lower bound, Ω(s(T )/n), for the generalization gap in non-smoothing settings when the training loss is 1/s(T ). Clearly, SGDmax does not achieve the lower bound. Therefore, we are motivated to design algorithms to reduce the non-vanishing sample size-independent term. The following main question of our paper arises: Can we eliminate the error floor in generalization bounds of adversarial generalization gap? We call the term c(T ) as generalization error floor. It is observed that the term c(T ) comes from the non-smoothness of the adversarial loss. Hence, stability analysis on some smoothed algorithms has been studied recently. It includes noise-SGD and differential privacy-SGD (Bassily et al., 2020) , adding noise to weight and data (Xing et al., 2021b) , stochastic weight averaging, and cyclic learning rate (Xiao et al., 2022) . Unfortunately, these smoothed algorithms cannot eliminate the generalization error floor. In this paper, we employ a smoothing technique using tools from Moreau envelope function to smooth the adversarial loss and perform gradient descent to this smooth surrogate. Following the name SGDmax, we refer the smoothed version of SGDmax as Smoothed-SGDmax, which improves adversarial generalization. We prove that Smoothed-SGDmax has the same training loss 1/s(T ) on adversarial loss. Most importantly, Smoothed-SGDmax eliminates the generalization error floor and achieves the minimax lower bound Ω(s(T )/n) of the generalization gap. The comparison of the stability-based generalization upper bound and lower bound of our proposed algorithm with the SGDmax-based adversarial training algorithm is given in Table 1 . Additionally, our proposed algorithm can be viewed as a general form of stochastic weight averaging (SWA (Izmailov et al., 2018) ). As a by-product, we provide an understanding of SWA in our framework, see more discussion in Sec. 5.4. In Fig. 1 , we show the training procedure of our proposed algorithm as well as adversarial training on CIFAR-10. The contributions of our work are listed as follows: 1. Main result: we prove that the generalization error floor in non-smooth loss minimization can be eliminated by a properly designed algorithm, which we called Smoothed-SGDmax. 2. We develop the minimax lower bound of the generalization gap in non-smooth loss minimization. Specifically, we show that an algorithm has at least Ω(s(T )/n) generalization gap if the training error training loss is 1/s(T ). Smoothed-SGDmax achieves the minimax lower bound. 3. Experiments on common datasets verify the theoretical results and show the effectiveness of our proposed algorithm in practice.



Figure 1: Experiments of adversarial training and Smoothed-SGDmax on CIFAR-10.

Comparison of stability-based generalization bounds of adversarial generalization gap. c 1 (T ) and c 2 (T ) are sample size-independent terms. Details of the form of s(T ), c 1 (T ), c 2 (T ) are discussed in Sec. 4 and Sec. 5.

