PointDP: DIFFUSION-DRIVEN PURIFICATION AGAINST 3D ADVERSARIAL POINT CLOUDS

Abstract

3D Point cloud is a critical data representation in many real-world applications, such as autonomous driving, robotics, and medical imaging. Although the success of deep learning further accelerates the adoption of 3D point clouds in the physical world, deep learning is notoriously vulnerable to adversarial attacks. Various defense solutions have been proposed to build robust models against adversarial attacks. In this work, we identify that the state-of-the-art empirical defense, adversarial training, has a major limitation in 3D point cloud models due to gradient obfuscation, resulting in significant degradation of robustness against strong attacks. To bridge the gap, we propose PointDP, a purification strategy that leverages diffusion models to defend against 3D adversarial attacks. Since PointDP does not rely on predefined adversarial examples for training, it can defend against diverse threats. We extensively evaluate PointDP on six representative 3D point cloud architectures and leverage sixteen strong and adaptive attacks to demonstrate its lower-bound robustness. Our evaluation shows that PointDP achieves significantly better (i.e., 12.6%-40.3%) adversarial robustness than state-of-the-art methods under strong attacks bounded by different ℓ p norms.

1. INTRODUCTION

Point cloud data is emerging as one of the most broadly used representations in 3D computer vision. It is a versatile data format available from various sensors like LiDAR and stereo cameras and computer-aided design (CAD) models, which depict physical objects by many coordinates in the 3D space. Many deep learning-based 3D perception models have been proposed [59, 34, 43, 60, 41, 9] and thus realized several safety-critical applications (e.g., autonomous driving) [81, 46, 45] . Although deep learning models [41, 42] have exhibited performance boost on many challenging tasks, extensive studies show that they are notoriously vulnerable to adversarial attacks [5, 49, 68] , where attackers manipulate the input in an imperceptible manner, which will lead to incorrect predictions of the target model. Because of the broad applications of 3D point clouds in safety-critical fields, it is imperative to study the adversarial robustness of point cloud recognition models. The manipulation space for 2D adversarial attacks is to change pixel-level numeric values of the input images. Unlike adversarial examples in 2D applications, the flexible representation of 3D point clouds results in an arguably larger attack surface. For example, adversaries could shift and detach existing points [88] , add new points into the pristine point cloud [50] , or even generate totally new point clouds [89] to launch attacks. Different strategies, including limits on the number of altered points and constraints on the maximal magnitude of shifted points [50] were proposed to make attacks less perceptible. The flexibility of 3D point cloud data formats enables diverse attacks, thus hindering a practical and universal defense design. Given the safety-critical property involved in 3D point cloud applications, various studies have been devoted to advancing the robustness of 3D point cloud recognition models. DUP-Net [90] and GvG-PointNet++ [14] pioneered to add statistical outlier removal (SOR) modules as pre-processing and in-network blocks, respectively, as mitigation strategies. More lately, Sun et al. In this work, our journey starts with revisiting the prior arts and exploring their truly adversarial robustness. By designing various types of strong adaptive attacks, we, for the first time, demonstrate that standard adversarial training [33] suffers from gradient obfuscation in the point cloud recognition models as the unstructured point cloud data format requires unique architectural designs to digest. We also extensively evaluate IF-Defense and LPC to show that their purification strategies are actually vulnerable to stronger attacks ( § 4.3). Furthermore, we propose PointDP, an adversarial purification method that leverages a diffusion model as a pre-processing module to defend against 3D adversaries. As shown in Figure 1 , PointDP consists of two components (1) an off-the-shelf 3D point cloud diffusion model and (2) a classifier. Given an input point cloud, PointDP take two steps: (i) adding noise to the input data gradually via the diffusion process of the diffusion model, (ii) purifying the noised data step by step to get the reversed sample via the reverse process of a diffusion model ( § 3.1), and (iii) feeding the reversed sample to the final classifier. Since PointDP does not rely on any types of pre-defined adversarial examples for training, it can defend against diverse unseen threats. We rigorously evaluate PointDP with six representative point cloud models and sixteen attacks, including PGD [50, 33] , C&W [68, 8] , and point cloud-specific attacks [88, 21] with ℓ 0 , ℓ 2 , and ℓ ∞ norms. PointDP on average achieves 75.9% robust accuracy while maintaining similar clean accuracy to the original models, outperforming existing studies by a significant margin. In a nutshell, our contributions are summarized as two-fold: • We are the first to demonstrate that standard adversarial training [33, 50] , the most longstanding defense in the 2D image recognition task, has a major limitation in its application in 3D point cloud models due to architecture designs. We launch black-box attacks to validate our claim that degrades adversarially trained models' robust accuracy to merely ∼10%, which is no longer useful for 3D point cloud recognition. • We propose PointDP that leverages diffusion models to purify adversarial 3D point clouds. PointDP is a general framework that is independent of the diffusion model used. We also formulate rigorous adaptive attacks on PointDP. We conduct extensive evaluation on six representative models with numerous attacks to comprehensively understand the robustness of PointDP. Our evaluation shows that PointDP outperforms previous state-of-the-arts purification methods, IF-Defense [66] and LPC [25] , by 12.6% and 40.3% on average, respectively. We also set up a rigorous protocol for 3D robustness evaluation to benefit future research.

2. RELATED WORK

In this section, we review the current progress of deep learning, adversarial attacks, and defenses for 3D point cloud recognition tasks.

2.1. DEEP LEARNING ON 3D POINT CLOUD RECOGNITION

2D computer vision has achieved stellar progress on architectural designs of convolutional neural networks [22], followed by vision transformers [15] . However, there is currently no consensus on the architecture of 3D perception models since there is no standard data format for 3D perception [53] . As raw data from both 3D scanners and triangular meshes can be efficiently transformed into point clouds, they are becoming the most often utilized data format in 3D perception. 3D networks at the early stage use dense voxel grids for perception [59, 34, 47, 54] 



[51]  broke the robustness of DUP-Net and GvG-PointNet++ by specific adaptive attacks. Adversarial training has been acknowledged as the most potent defense to deliver strong empirical robustness on PointNet,DGCNN, and PCT [50]. Meanwhile, advanced purification strategies like IF-Defense [66] and LPC [25] leverage more complex modules to cleanse the adversarial point clouds. However, given that point cloud is a sparse and unstructured data format, it motivates us to re-think that whether the current adversarial training and purification-based methods are robust enough against stronger adversarial attacks?

, which discretize a point cloud to voxel cells for classification, segmentation, and object detection. PointNet pioneered to leverage global pooling help achieve memory-efficient permutation invariance in an end-to-end manner.PointNet++ [42]  and DGCNN [61] followed up to add sophisticated local clustering operations to advance the performance. Sparse tensors are the other direction in 3D network designs[19, 9]  to use 3D convolutions to improve 3D perception performance. PointCNN and RSCNN reformed the classic pyramid CNN to improve the local feature generation [26, 29]. PointConv and KPConv designed new convolution operation for point cloud learning [65, 55]. PointTransformer and PCT advanced self-attention blocks in the 3D space and achieved good performance [87, 20]. Various novel local clustering operations [69, 32] also show enhancements on the clean performance. In this work, we focus on PointNet, PointNet++, DGCNN, PCT, CurveNet, and PointMLP as our evaluation backbones since they are representative and widely used and achieve state-of-the-art results in point cloud recognition [1].

