DeepReflecs: Deep Learning for Automotive Object Classification with Radar Reflections

This paper presents an novel object type classification method for automotive applications which uses deep learning with radar reflections. The method provides object class information such as pedestrian, cyclist, car, or non-obstacle. The method is both powerful and efficient, by using a light-weight deep learning approach on reflection level radar data. It fills the gap between low-performant methods of handcrafted features and high-performant methods with convolutional neural networks. The proposed network exploits the specific characteristics of radar reflection data: It handles unordered lists of arbitrary length as input and it combines both extraction of local and global features. In experiments with real data the proposed network outperforms existing methods of handcrafted or learned features. An ablation study analyzes the impact of the proposed global context layer.

[1]  Wolfram Burgard,et al.  Holistic Filter Pruning for Efficient Deep Neural Networks , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  Klaus C. J. Dietmayer,et al.  Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges , 2019, IEEE Transactions on Intelligent Transportation Systems.

[3]  Klaus Dietmayer,et al.  Using Machine Learning to Detect Ghost Images in Automotive Radar , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[4]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Di Feng,et al.  Leveraging Uncertainties for Deep Multi-modal Object Detection in Autonomous Driving , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[6]  Dariu M. Gavrila,et al.  CNN Based Road User Detection Using the 3D Radar Cube , 2020, IEEE Robotics and Automation Letters.

[7]  Wolfram Burgard,et al.  Learning multimodal fixed-point weights using gradient descent , 2019, ESANN.

[8]  Nils Appenrodt,et al.  Radar-based Road User Classification and Novelty Detection with Recurrent Neural Network Ensembles , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[9]  Ingo Weber,et al.  Instantaneous Ghost Detection Identification in Automotive Scenarios , 2019, 2019 IEEE Radar Conference (RadarConf).

[10]  Bin Yang,et al.  Deep Learning-based Object Classification on Automotive Radar Spectra , 2019, 2019 IEEE Radar Conference (RadarConf).

[11]  Bin Yang,et al.  Short-Duration Doppler Spectrogram for Person Recognition with a Handheld Radar , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[12]  Bin Yang,et al.  Person Recognition Based on Micro-Doppler and Thermal Infrared Camera Fusion for Firefighting , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[13]  Jürgen Dickmann,et al.  Semantic Segmentation on Radar Point Clouds , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[14]  Marcel Hoffmann,et al.  Pedestrian Classification with a 79 GHz Automotive Radar Sensor , 2018, 2018 19th International Radar Symposium (IRS).

[15]  Yilong Lu,et al.  Classification and Tracking of Moving Objects from 77 GHz Automotive Radar Sensors , 2018, 2018 IEEE International Conference on Computational Electromagnetics (ICCEM).

[16]  Urs Schneider,et al.  Micro-doppler based human-robot classification using ensemble and deep learning approaches , 2017, 2018 IEEE Radar Conference (RadarConf18).

[17]  Thomas Wagner,et al.  Radar Signal Processing for Jointly Estimating Tracks and Micro-Doppler Signatures , 2017, IEEE Access.

[18]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Markus Hahn,et al.  Potential of radar for static object classification using deep learning methods , 2016, 2016 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[20]  Yimin Zhang,et al.  Radar Signal Processing for Elderly Fall Detection: The future for in-home monitoring , 2016, IEEE Signal Processing Magazine.

[21]  Ricardo Omar Chávez García,et al.  Multiple Sensor Fusion and Classification for Moving Object Detection and Tracking , 2016, IEEE Transactions on Intelligent Transportation Systems.

[22]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[23]  Sun Hongbin,et al.  Frontal object perception for Intelligent Vehicles based on radar and camera fusion , 2016, 2016 35th Chinese Control Conference (CCC).

[24]  Jürgen Dickmann,et al.  Detection of parked vehicles from a radar based occupancy grid , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[25]  Igal Bilik,et al.  Application of a 24 GHz FMCW automotive radar for urban target classification , 2014, 2014 IEEE Radar Conference.

[26]  Ralph Helmar Rasshofer,et al.  Pedestrian recognition using automotive radar sensors , 2012 .

[27]  H. Rohling,et al.  Pedestrian classification in automotive radar systems , 2012, 2012 13th International Radar Symposium.

[28]  Henning Ritter,et al.  Pedestrian detection procedure integrated into an 24 GHz automotive radar , 2010, 2010 IEEE Radar Conference.

[29]  Tarak Gandhi,et al.  Pedestrian Protection Systems: Issues, Survey, and Challenges , 2007, IEEE Transactions on Intelligent Transportation Systems.

[30]  M.-M. Meinecke,et al.  Radar sensors and sensor platform used for pedestrian protection in the EC-funded project SAVE-U , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[31]  S Milch,et al.  PEDESTRIAN DETECTION WITH RADAR AND COMPUTER VISION , 2001 .

[32]  Donald E. Barrick,et al.  FM/CW Radar Signals and Digital Processing , 1973 .