CNN Based Road User Detection Using the 3D Radar Cube

This letter presents a novel radar based, single-frame, multi-class detection method for moving road users (pedestrian, cyclist, car), which utilizes low-level radar cube data. The method provides class information both on the radar target- and object-level. Radar targets are classified individually after extending the target features with a cropped block of the 3D radar cube around their positions, thereby capturing the motion of moving parts in the local velocity distribution. A Convolutional Neural Network (CNN) is proposed for this classification step. Afterwards, object proposals are generated with a clustering step, which not only considers the radar targets’ positions and velocities, but their calculated class scores as well. In experiments on a real-life dataset we demonstrate that our method outperforms the state-of-the-art methods both target- and object-wise by reaching an average of 0.70 (baseline: 0.68) target-wise and 0.56 (baseline: 0.48) object-wise F1 score. Furthermore, we examine the importance of the used features in an ablation study.

[1]  Wolfgang Menzel,et al.  High resolution automotive radar measurements of vulnerable road users – pedestrians & cyclists , 2015, 2015 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[2]  Markus Hahn,et al.  Potential of radar for static object classification using deep learning methods , 2016, 2016 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[3]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[4]  Wolfgang Menzel,et al.  A multi-reflection-point target model for classification of pedestrians by automotive radar , 2014, 2014 11th European Radar Conference.

[5]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[6]  Jürgen Dickmann,et al.  Comparison of random forest and long short-term memory network performances in classification tasks using radar , 2017, 2017 Sensor Data Fusion: Trends, Solutions, Applications (SDF).

[7]  Nojun Kwak,et al.  Human detection by Neural Networks using a low-cost short-range Doppler radar sensor , 2017, 2017 IEEE Radar Conference (RadarConf).

[8]  Jürgen Dickmann,et al.  Supervised Clustering for Radar Applications: On the Way to Radar Instance Segmentation , 2018, 2018 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[9]  Dave Tahmoush,et al.  Radar micro-doppler for long range front-view gait recognition , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[10]  Dariu Gavrila,et al.  EuroCity Persons: A Novel Benchmark for Person Detection in Traffic Scenes , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Nils Appenrodt,et al.  Radar-based Road User Classification and Novelty Detection with Recurrent Neural Network Ensembles , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[12]  Dariu M. Gavrila,et al.  Occlusion aware sensor fusion for early crossing pedestrian detection , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[13]  Jürgen Dickmann,et al.  Radar-based Feature Design and Multiclass Classification for Road User Recognition , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[14]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[16]  Jürgen Dickmann,et al.  Semantic Segmentation on Radar Point Clouds , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[17]  Wolfgang Menzel,et al.  Clustering of high resolution automotive radar detections and subsequent feature extraction for classification of road users , 2015, 2015 16th International Radar Symposium (IRS).

[18]  Marcel Hoffmann,et al.  Pedestrian Classification with a 79 GHz Automotive Radar Sensor , 2018, 2018 19th International Radar Symposium (IRS).

[19]  Paul Newman,et al.  Probably Unknown: Deep Inverse Sensor Modelling Radar , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[20]  Takuro Sato,et al.  Technique of tracking multiple pedestrians using monostatic ultra-wideband Doppler radar with adaptive Doppler spectrum estimation , 2016, 2016 International Symposium on Antennas and Propagation (ISAP).

[21]  Marcel Hoffmann,et al.  Image-Based Pedestrian Classification for 79 GHz Automotive Radar , 2018, 2018 15th European Radar Conference (EuRAD).

[22]  Francesco Fioranelli,et al.  Practical classification of different moving targets using automotive radar and deep neural networks , 2018, IET Radar, Sonar & Navigation.

[23]  Nils Appenrodt,et al.  A Multi-Stage Clustering Framework for Automotive Radar Data , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[24]  Dariu Gavrila,et al.  SafeVRU: A Research Platform for the Interaction of Self-Driving Vehicles with Vulnerable Road Users , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[25]  H. Hirschmüller Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Stereo Processing by Semi-global Matching and Mutual Information , 2022 .

[26]  Erwin Biebl,et al.  Single-Frame Vulnerable Road Users Classification with a 77 GHz FMCW Radar Sensor and a Convolutional Neural Network , 2018, 2018 19th International Radar Symposium (IRS).

[27]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[28]  Louis B. Rall,et al.  Automatic differentiation , 1981 .

[29]  Karl Granström,et al.  Extended Object Tracking: Introduction, Overview and Applications , 2016, ArXiv.

[30]  Klaus C. J. Dietmayer,et al.  2D Car Detection in Radar Data with PointNets , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[31]  Raja Giryes,et al.  Deep Radar Detector , 2019, 2019 IEEE Radar Conference (RadarConf).

[32]  Bin Yang,et al.  Deep Learning-based Object Classification on Automotive Radar Spectra , 2019, 2019 IEEE Radar Conference (RadarConf).