TRANSFERABLE UNLEARNABLE EXAMPLES

Abstract

With more people publishing their personal data online, unauthorized data usage has become a serious concern. The unlearnable examples strategies have been introduced to prevent third parties from training on the data without permission. They add perturbations to the users' data before publishing, so as to make the models trained on the perturbed published dataset invalidated. These perturbations have been generated for a specific training setting and a target dataset. However, their unlearnable effects significantly decrease when used in other training settings or datasets. To tackle this issue, we propose a novel unlearnable strategy based on Class-wise Separability Discriminant (CSD), which boosts the transferability of the unlearnable perturbations by enhancing the linear separability. Extensive experiments demonstrate the transferability of the unlearnable examples crafted by our proposed method across training settings and datasets. The implementation of our method is available at https://github.com/renjie3/TUE.

1. INTRODUCTION

With more people posting their personal data online (intentionally or unintentionally), it has raised the concern that the data might be utilized without the owner's consent to train commercial or malicious machine learning models. While large-scale datasets collected from the Internet like LFW (Huang et al., 2008 ), Freebase(Bollacker et al., 2008 ), and Ms-celeb-1m(Guo et al., 2016) have greatly advanced the development of deep learning, they may contain a certain amount of private data, which has the potential risk of privacy leakage. Thus, growing efforts (Huang et al., 2020; Fowl et al., 2021) have been made to protect data from unauthorized usage by making the data samples "unlearnable" (Huang et al., 2020; Fowl et al., 2021; He et al., 2022) . These methods generate the unlearnable examples by injecting imperceptible "shortcut" perturbation. If the data is used by unauthorized training, the models will be tricked to extract such easy-to-learn shortcut features and ignore the real semantics in the original data (Geirhos et al., 2020) . Consequently, the trained model fails to recognize the user's data during the test phase and the user's data gets protected. As shown in Figure 1 , the practical usage procedure of unlearnable examples (UEs) consists of two stages, i.e., the generation stage and the evaluation stage. Before releasing data, in the generation stage, one can make the data unlearnable by adding perturbations into original data. According to whether the label information is used to generate the perturbations, we divide the UEs into Supervised UEs which is generated with the guidance of the label information, and Unsupervised UEs which is generated without the guidance of label information. In the evaluation stage, the unlearnable version of data is released to the public, and the unauthorized third parties might use different algorithms, like supervised training and unsupervised training, to learn from the data. The UEs aim to provide protection in this stage against unauthorized training and invalidate the models trained on them. However, existing UEs are in lack of training-wise transferability and have weaknesses in datawise transferability in the evaluation stage. First, low training-wise transferability implies that the perturbed samples generated towards one target training setting cannot be transferred to other * Equal contribution. 1

