Privacy-preserving Learning via Deep Net Pruning

Abstract

Neural network pruning has demonstrated its success in significantly improving the computational e ciency of deep models while only introducing a small reduction on final accuracy. In this paper, we explore an extra bonus of neural network pruning in terms of enhancing privacy. Specifically, we show a novel connection between magnitude-based pruning and adding di↵erentially private noise to intermediate layers under the overparameterized regime. To the best of our knowledge, this is the first work that bridges pruning with the theory of di↵erential privacy. The paper also presents experimental results by running the model inversion attack on two benchmark datasets, which supports the theoretical finding.

1. Introduction

Data privacy has become one of the top concerns in the application of deep neural networks, since there has been an increasing demand to train deep models on private data sets. 2020). However, the majority of existing literature only demonstrate the benefits of pruning in terms of energy saving and inference speedup, while in this work, we investigate another interesting bonus of pruningpreserving data privacy. Our investigation is mainly inspired by the observation that neural network pruning makes the inversion from hidden-layers harder, as the percentage of remained weight decreases (see Figure 1 ). Motivated by this empirical observation, we build under the over-paramterized regime of deep learning theory, and show an interesting connection between neural network pruning and adding di↵erentially private noise to intermediate layers. We believe this con- 2016) using the inversion algorithm in Section 5.2. We prune the network with di↵erent k's, the fraction of remained weights. Inverted images from networks with fewer weights are visually more di↵erent from the original image. nection may have important practical implications since the pruned model only incurs small accuracy loss, and we leave that as future work. We list our contributions as follow: • We explore the benefits of pruned neural networks in terms of preserving data privacy. To the best of our knowledge, this is the first step towards drawing a theoretical connection between neural network pruning and di↵erential privacy. • To build the connection between pruning and adding di↵erentially private noise to intermediate layers, we generalize the famous anti-concentration inequality due to Carbery and Wright Carbery & Wright (2001) . This generalization might find more applications in the theoretical analysis of neural network pruning in the future. • We provide empirical results in support of our theoretical finding. Specifically, we demonstrate on two benchmark datasets that pruned neural networks are more secure in the sense that running the model inversion attack becomes harder. Roadmap. The rest of this paper is organized as follow. Section 2 covers existing literature in di↵erent privacy, neural network pruning, and over-parameterized deep learning theory. Section 3 provides theoretical preliminaries and Section 4 presents our main theoretical result. Section 5 shows empirical results on MNIST and CIFAR-10 benchmarks that are in support of our theoretical findings. We conclude this work in Section 6.

2. Related work

Neural network pruning Traditional deep neural network models are computationally expensive and memory intensive, which hinders their deployment in applications with limited memory resources or strict latency requirements. Although there are existing studies about applying di↵erential privacy in neural network, but there is little exploration on presenting di↵erential privacy using prune network yet.



For example, hospitals are now training their automated diagnosis systems on private patients' data Litjens et al. (2016); Lakhani & Sundaram (2017); De Fauw et al. (2018); and advertisement providers are collecting users' online trajectories to optimize their learning-based recommendation algorithm Covington et al. (2016); Ying et al. (2018). These private data, however, are usually subject to the regulations such as California Consumer Privacy Act (CCPA) Legislature (2018), Health Insurance Portability and Accountability Act (HIPAA) Act (1996), and General Data Protection Regulation (GDPR) of European Union. Di↵erential privacy (DP) Dwork et al. (2006b); Dwork (2009); Dwork & Roth (2014) has emerged, during the past few years, as a strong standard to provide theoretical privacy guarantees for algorithms on aggregate databases. The core idea of achieving di↵erential privacy is to add controlled noise to the output of a deterministic function, such that the output cannot be used to infer much about any single individual in the database. Recent years have seen an increasing number of applications that adapt di↵erential privacy mechanisms to address privacy concerns in deep learning Shokri & Shmatikov (2015); Abadi et al. (2016); Phan et al. (2016); McMahan et al. (2018). Neural network pruning (or pruning in short), a seemingly orthogonal field to privacy, has also been the subject of a great amount of work in recent years. Pruning aims to reduce the number of model parameters, such that the compressed model can be applied even under the memory constraints of the edge-devices. Various pruning techniques have succeeded in significantly compressing models with little or no loss of accuracy Han et al. (2015; 2016a); Li et al. (2016); Ding et al. (2018); Evci et al. (2019); Tanaka et al. (

Figure 1: Visualization of inverted CIFAR-10 Krizhevsky (2009) inputs from the third bottleneck of ResNet-18 He al. (2016) using the inversion algorithm in Section 5.2. We prune the network with di↵erent k's, the fraction of remained weights. Inverted images from networks with fewer weights are visually more di↵erent from the original image.

Among them, neural network pruning has been widely adopted because it is able to reduce model sizes by up to one order of magnitude without significant accuracy loss. The idea of network pruning dates back to the Optimal Brain Damage in 1990sLeCun  et al. (1990). Recently, it has been shown that removing the weights with low magnitude can also achieve a highly compressed modelHan et al. (2016b), which is referred to as 'magnitude-based pruning'. Di↵erential privacy has been successfully applied to many problems. For more detailed surveys of the applications of di↵erential privacy, we refer the readers toDwork  (2008; 2011). Applying di↵erential privacy techniques in deep learning is an interesting but non-trivial task. Previous research have customized di↵erential privacy for di↵erent learning tasks and settingsShokri & Shmatikov (2015);Abadi et al. (2016); Phan et al. (2016).

