ADAPTIVE AUTOMOTIVE RADAR DATA ACQUISITION

Abstract

In an autonomous driving scenario, it is vital to acquire and efficiently process data from various sensors to obtain a complete and robust perspective of the surroundings. Many studies have shown the importance of having radar data in addition to images since radar improves object detection performance. We develop a novel algorithm motivated by the hypothesis that with a limited sampling budget, allocating more sampling budget to areas with the object as opposed to a uniform sampling budget ultimately improves relevant object detection and classification. In order to identify the areas with objects, we develop an algorithm to process the object detection results from the Faster R-CNN object detection algorithm and the previous radar frame and use these as prior information to adaptively allocate more bits to areas in the scene that may contain relevant objects. We use previous radar frame information to mitigate the potential information loss of an object missed by the image or the object detection network. Also, in our algorithm, the error of missing relevant information in the current frame due to the limited budget sampling of the previous radar frame did not propagate across frames. We also develop an end-to-end transformer-based 2D object detection network using the NuScenes radar and image data. Finally, we compare the performance of our algorithm against that of standard CS and adaptive CS using radar on the Oxford Radar RobotCar dataset.

1. INTRODUCTION

The intervention of deep learning and computer vision techniques for autonomous driving scenario is aiding in the development of robust and safe autonomous driving systems. Similar to humans navigating their world with numerous sensors and information, the autonomous driving systems need to process different sensor information efficiently to obtain the complete perspective of the environment to safely maneuver. Numerous studies Meyer & Kuschk (2019) , Chang et al. (2020) have shown the importance of having radar data in addition to images for improved object detection performance. The real-time radar data acquisition using compressed sensing is a well-studied field where, even with sub-Nyquist sampling rates, the original data can be reconstructed accurately. During the onboard signal acquisition and processing, compressed sensing will reduce the required measurements, therefore, gaining speed and power savings. In adaptive block-based compressed sensing, based on prior information with a limited sampling budget, radar blocks with objects would be allocated more sampling resources while maintaining the overall sampling budget. This method would further enhance the quality of reconstructed data by focusing on the important regions. In our work, we split the radar into 8 azimuth blocks and used the 2D object detection results from images as prior data to choose the important regions. The 2D object detection network generates the bounding boxes and object classes for objects in the image. The bounding boxes were used to identify the azimuth of the object in radar coordinates. This helped in determining the important azimuth blocks. As a second step, we used both previous radar information and the 2-D object detection network to determine the important regions and dynamically allocate the sampling budget. The use of previous radar data in addition to object information from images mitigates the loss of object information either by image or the object detection network. • We have developed an algorithm in a limited sampling budget environment to dynamically allocate more sampling budget to important radar return regions based on the prior image data processed by the Faster R-CNN object detection network Ren et al. ( 2015) while maintaining the overall sampling budget. • In the extended algorithm, important radar return regions are selected using both object detection output and the previous radar frame. 

2. RELATED WORK

The compressed sensing technique is a well-studied method for sub-Nyquist signal acquisition. The Adaptive compressed sensing technique was used for radar acquisition by Assem et al. (2016) . They used the previously received pulse interval as prior information for the present interval to determine the important regions of the pulsed radar. In our first algorithm, we used only the image data and in the second algorithm, we used both image and previous radar data on the FMCW scanning radar. In another work, Kyriakides (2011) they used adaptive compressed sensing for a static tracker case and have shown improved target tracking performance. However, in our algorithm, we've used adaptive compressed sensing for radar acquisition from an autonomous vehicle where, both the vehicle and the objects were moving. In Zhang et al. ( 2012) they used adaptive compressed sensing by optimizing the measurement matrix as a separate least squares problem, in which only the targets are moving. This increases the computational complexity of the overall algorithm. Whereas in our method, the measurement matrix size is increased to accommodate more sampling budget which has the same complexity as the original CS measurement matrix generation technique. In a separate but 



, we have also developed an end-to-end transformer-based 2-D object detection Carion et al. (2020) network using the NuScenes Caesar et al. (2020) radar and image dataset. The object detection performance of the model using both Image and Radar data performed better than the object detection model trained only on the image data.

We've designed an end-to-end transformer-based 2-D object detection network (DETR-Radar) Carion et al. (2020) using both Nuscenes Caesar et al. (2020) radar and image data.

related work, Nguyen et al. (2019)  LiDAR data was acquired based on Region-of-Interest derived from image segmentation results of images. In our work, we acquired radar data using 2-D object detection results.Apart from radar, adaptive CS was used for images and videos. InMehmood et al. (2017), the spatial entropy of the image helped in determining the important regions. The important regions were then allocated more sampling budget than the rest which improved reconstruction quality. In another work, Zhu et al. (2014), the important blocks were determined based on the variance of each block, the entropy of each block and the number of significant Discrete Cosine Transform (DCT) coefficients in the transform domain. In Wells & Chatterjee (2018), object tracking across frames of a video was preformed using Adaptive CS. The background and foreground segmentation helped in allocating higher sampling density to the foreground and very low sampling density to the background region.Liu et al. (2011)  similarly used adaptive CS for video acquisition based on inter-frame correlation. Also,Ding et al. (2015), performed the joint CS of two adjacent frames in a video based on the correlation between the frames. Finally, numerous studies showed the advantage of using both radar and images in an object detection network for improved object detection performance. Nabati & Qi (2019) used radar points to generate region proposals for the Fast R-CNN object detection network which made the model faster than the selective search based region proposal algorithm in Fast R-CNN. Nobis et al. (2019),  Chadwick et al. (2019)  showed that radar in addition to image improved distant vehicle object detec-

InRoos et al. (2018), they showed the successful application of standard compressed sensing on radar data with 40% sampling rate. They also showed that they could use a single A/D converter for multiple antennas. In our method, we have shown efficient reconstruction with 10% sampling rate. In another work, Slavik et al. (2016), they used standard compressed sensing-based signal acquisition for noise radar with 30% sampling rate. Whereas, we've developed this algorithm for FMCW scanning radar.Correas-Serrano & González-Huici (2018)  analyzed various compressed sensing reconstruction algorithms such as Orthogonal Matching Pursuit (OMP), Basis Pursuit De-noising

