DP-SGD-LF: IMPROVING UTILITY UNDER DIFFEREN-TIALLY PRIVATE LEARNING VIA LAYER FREEZING

Abstract

Differentially Private SGD (DP-SGD) is a widely known substitute for SGD to train deep learning models with privacy guarantees. However, privacy guarantees come at cost in model utility. The key DP-SGD steps responsible for this utility cost are per-sample gradient clipping, which introduces bias, and adding noise to the aggregated (clipped) gradients, which increases the variance of model updates. Inspired by the observation that different layers in a neural network often converge at different rates following a bottom-up pattern, we incorporate layer freezing into DP-SGD to increase model utility at fixed privacy budget. Through theoretical analysis and empirical evidence we show that layer freezing improves model utility, by reducing both the bias and variance introduced by gradient clipping and noising. These improvements in turn lead to better model accuracy, and empirically generalize over multiple datasets, models, and privacy budgets.

1. INTRODUCTION

Deep Neural Networks (DNNs) have seen a growing success at many tasks under various domains in recent years. As a result DNNs are now deployed in numerous applications, including some involving sensitive data, such as users' medical history, purchasing records, or chat histories. In these sensitive applications, data privacy is a concern. However, there is strong evidence that deep learning models memorize, and thus leak, information about their training data (Shokri et al., 2016; Carlini et al., 2020; 2022; Feldman & Zhang, 2020) . To prevent data leakage, common DNN training algorithms such as Stochastic Gradient Descent (SGD) and its variants have been adapted to enforce Differential Privacy (DP) Song et al. (2013); Dwork et al. (2006) , a rigorous privacy guarantee which provably mitigates data leaks. As a convenient drop-in replacement for SGD, the DP-SGD algorithm is commonly used for privacy-preserving machine learning, and numerous efforts haave improved its theoretical privacy analysis (Abadi et al., 2016; Mironov, 2017; Mironov et al., 2019) . However, the privacy guarantees offered by DP-SGD still come at a substantial cost in model utility (accuracy), despite substantial practical improvements over time De et al. (2022) ; Papernot et al. (2021) . There are two key changes to SGD that DP-SGD introduces in each model update step. Each change is required to prove privacy guarantees, and contributes to utility costs. The first change is to clip per-sample gradient to a fixed L 2 norm bound, which introduces bias in the estimation of the gradient descent direction. The second change is to add noise from a standard Gaussian to the aggregated (clipped) gradients, which increases the variance of model updates. We show using theoretical analysis that increasing the gradient clipping norm of a given DNN layer in DP-SGD reduces the variance introduced by DP noise and, under some assumptions, the clipping bias as well. Both lead to better convergence upper-bounds for DP-SGD. We combine this result with the observation that different layers in a DNN trained with SGD converge at different rates following a bottom-up pattern-which we empirically verify also holds for DP-SGD-and introduce the DP-SGD Layer Freeze (DP-SGD-LF) algorithm. This algorithm freezes the lower layers (closer to the input) of a DNN towards the end of training, which increases the norm of clipped gradients for the remaining layers, thereby decreasing the bias and variance introduced by DP-SGD when updating these parameters. Since the remaining layers benefit more from updates at this point of traininig, the finial accuracy increases. 



We apply DP-SGD-LF to state of the art DP-SGD implementations on three datasets De et al. (2022); Papernot et al. (2021), and show that it improves the final model's accuracy by up to 1.3 percentage

annex

points, and is particularly effective in the high privacy (low DP ϵ) regime. We also show that DP-SGD-LF is not sensitive to hyper-parameters, and propose and use easy to set, reasonable defaults.The rest of the paper describes our contributions: after introducing the necessary background in §2, §3 introduces our algorithm, and supports its design through empirical and theoretical analysis. §4 then empirically confirms the expected behavior, and shows that DP-SGD-LF improves the accuracy of different models over multiple image classification datasets.

2. BACKGROUND

Mini-batch SGD is one of the most commonly used optimization algorithm in non-private deep learning. For each iteration t, and calling η t is the step size, SGD updates the parameters of the model θ by stepping into the direction of steepest descent, estimated with the averaged gradients over B samples in a mini-batch,The convergence analysis of the SGD algorithm often rely on the following fundamental result. Lemma 2.1 (Decent Lemma (Bottou et al., 2018) ). Assuming the objective function f : R d -→ R to be continuously differentiable and the gradient of f , ∇f :Under privacy constraints, the DP-SGD algorithm provides a convenient substitution to SGD for training DNNs with differential privacy guarantees (Abadi et al., 2016) . The DP-SGD algorithm protects privacy by clipping the per-sample gradient vector, g t (x i ) ← -∇ θt f (θ t , x i ), and adding noise drawn from a Normal distribution to the aggregated clipped gradients. Let C be the L2-norm clipping threshold, σ be the noise multiplier, and d be the dimension of the model's parameters. the update rule for DP-SGD in each iteration is:where C controls the maximum influence that an individual sample can have on the gradient (the sensitivity), and σ controls the noise level scaled with respect to the sensitivity. We use the analysis based on Rényi Differential Privacy (RDP) (Mironov, 2017) for privacy accounting. The composition over t steps of training and the conversion of the RDP guarantee to the (ϵ, δ)-DP guarantee follow from the results in Mironov et al. (2019) . we use the publicly available implementation of the RDP privacy accountant in Opacus (Yousefpour et al., 2021) .

3. DIFFERENTIALLY PRIVATE LEARNING WITH LAYER FREEZING

We propose to incorporate layer freezing with DP-SGD, and demonstrate its effectiveness in increasing the trained model's predictive accuracy at fixed privacy budget. The intuition behind the performance gain is as follows. The two key steps in DP-SGD, clipping and noising, provide a DP guarantee at the cost of degrading model utility: clipping potentially introduces bias into the estimated descent direction, since it truncates individual gradients before aggregation to control sensitivity (Chen et al., 2021; Pichapati et al., 2019; Zhang et al., 2019) ; noising introduces variance on top of the biased estimate by adding random noise to the aggregated clipped gradients. Freezing parameters limits the model capacity in learning representations, but could bring benefits by reducing the bias and variance caused by clipping and noising on the remaining trainable parameters. Given the observation that lower layers (closer to the input side) converge faster than higher layers (closer to the prediction), we can freeze the parameters in lower layers during training, to minimally sacrifice model capacity in exchange for the benefits of better updates for the upper layer parameters.

