ON THE SYSTEM-LEVEL EFFECTIVENESS OF PHYSICAL OBJECT-HIDING ADVERSARIAL ATTACK IN AUTONOMOUS DRIVING

Abstract

In Autonomous Driving (AD) systems, perception is crucial for both security and safety. Among the different attacks on AD perception, the physical object-hiding adversarial attacks are especially severe due to their direct impact on road safety. However, we find that all existing works so far only evaluate their attack effect at the targeted AI component level, without any evaluation at the system level, i.e., with the entire system semantics and context such as the full AD system pipeline and closed-loop control. This thus inevitably raise a critical research question: can these existing research efforts actually effectively achieve the desired system-level attack effects (e.g., causing vehicle collisions, traffic rule violations, etc.) in the real-world AD system context? In the paper, we perform the first measurement study on whether and how effective the existing designs can lead to system-level effects, where we take the STOP sign-hiding attack as our target. Our evaluation results show that all the representative prior works cannot achieve any systemlevel effect in a classical closed-loop AD setup in road speeds controlled by common STOP signs. We then point out two limitation hypotheses that appear in all existing works: 1) the unpractical STOP sign size distribution in pixel sampling, and 2) missing particular consideration in system-critical attack range. Our results demonstrate that after overcoming these two limitations, the system-level effects can be further improved, i.e., the violation rate can increase around 70%.

1. INTRODUCTION

Autonomous Driving (AD) vehicles are now a reality in our daily life, where a wide variety of commercial and private AD vehicles are driving on the road. For example, the millions of Tesla cars (Kane, 2021) are equipped with Autopilot (Tesla, 2022). To ensure safe and correct driving, a fundamental pillar in the AD system is perception, which is designed to detect surrounding objects in real time. Due to the direct impact on safety-critical driving decisions such as collision avoidance, various prior works have studied the security of AD perception, especially the ones that aim at causing the disappearance of critical physical road objects (e.g., STOP signs), or physical objecthiding adversarial attacks (Jia et al., 2022; Xu et al., 2020; Chen et al., 2018; Wu et al., 2020) . Although a plethora of prior works studied such physical object-hiding adversarial attacks in AD settings, we find that all of them only evaluate their attack effect at the targeted AI component level (i.e., judged by per-frame object misdetection rates (Chen et al., 2018; Eykholt et al., 2018; Xu et al., 2020; Zhao et al., 2019; Jia et al., 2022) ), without any evaluation at the system level, i.e., with the full system semantics and context enclosing such AI component (e.g., the remaining AD system pipeline such as object tracking, planning, and control, closed-loop control, and the attack-targeted driving scenario), which we call the system model for such adversarial attacks in this paper ( §2). This thus inevitably raises a critical research question: can these existing works on physical objecthiding adversarial attacks effectively achieve the desired system-level attack effects (e.g., causing vehicle collisions, traffic rule violations, etc.) in the real-world AD system context? To systematically answer this critical research question, we take the necessary first step by performing a measurement study on prior works with regard to their capabilities in causing system-level effects. We select STOP sign-hiding attack as our target considering its high representativeness in physical object-hiding adversarial attack today (Shen et al., 2022) , and its direct impacts on driving correctness and road safety. We first classify the existing STOP sign-hiding adversarial attacks

