Deep Instance Segmentation With Automotive Radar Detection Points

Automotive radar provides reliable environmental perception in all-weather conditions with affordable cost, but it hardly supplies semantic and geometry information due to the sparsity of radar detection points. With the development of automotive radar technologies in recent years, instance segmentation becomes possible by using automotive radar. Its data contain contexts such as radar cross section and micro-Doppler effects, and sometimes can provide detection when the field of view is obscured. The outcome from instance segmentation could be potentially used as the input of trackers for tracking targets. The existing methods often utilize a clustering-based classification framework, which fits the need of real-time processing but has limited performance due to minimum information provided by sparse radar detection points. In this paper, we propose an efficient method based on clustering of estimated semantic information to achieve instance segmentation for the sparse radar detection points. In addition, we show that the performance of the proposed approach can be further enhanced by incorporating the visual multi-layer perceptron. The effectiveness of the proposed method is verified by experimental results on the popular RadarScenes dataset, achieving 89.53% mean coverage and 86.97% mean average precision with the IoU threshold of 0.5, which is superior to other approaches in the literature. More significantly, the consumed memory is around 1MB, and the inference time is less than 40 ms, indicating that our proposed algorithm is storage and time efficient. These two criteria ensure the practicality of the proposed method in real-world systems.

[1]  Shi-Min Hu,et al.  Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ping Luo,et al.  CycleMLP: A MLP-like Architecture for Dense Prediction , 2021, ICLR.

[3]  Nenghai Yu,et al.  CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Xinggang Wang,et al.  Hierarchical Aggregation for 3D Instance Segmentation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Sihan Chen,et al.  Radar Transformer: An Object Classification Network Based on 4D MMW Imaging Radar , 2021, Sensors.

[6]  Quoc V. Le,et al.  Pay Attention to MLPs , 2021, NeurIPS.

[7]  A. Dosovitskiy,et al.  MLP-Mixer: An all-MLP Architecture for Vision , 2021, NeurIPS.

[8]  Jürgen Dickmann,et al.  RadarScenes: A Real-World Radar Point Cloud Data Set for Automotive Applications , 2021, 2021 IEEE 24th International Conference on Information Fusion (FUSION).

[9]  Jenq-Neng Hwang,et al.  RODNet: Radar Object Detection using Cross-Modal Supervision , 2021, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Chen Feng,et al.  3D Point Cloud Processing and Learning for Autonomous Driving: Impacting Map Creation, Localization, and Perception , 2021, IEEE Signal Processing Magazine.

[11]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[12]  F. Tupin,et al.  CARRADA Dataset: Camera and Automotive Radar with Range- Angle- Doppler Annotations , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[13]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  C. Waldschmidt,et al.  Automotive Radar — From First Efforts to Future Systems , 2021, IEEE Journal of Microwaves.

[15]  Bharanidhar Duraisamy,et al.  Detection and Tracking on Automotive Radar Data with Deep Learning , 2020, 2020 IEEE 23rd International Conference on Information Fusion (FUSION).

[16]  B. Sick,et al.  Off-the-shelf sensor vs. experimental radar - How much resolution is necessary in automotive radar classification? , 2020, 2020 IEEE 23rd International Conference on Information Fusion (FUSION).

[17]  Siheng Chen,et al.  3D Point Cloud Processing and Learning for Autonomous Driving , 2020, ArXiv.

[18]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Ingo Weber,et al.  Semantic Segmentation on 3D Occupancy Grids for Automotive Radar , 2020, IEEE Access.

[20]  Siyang Cao,et al.  Robust and Adaptive Radar Elliptical Density-Based Spatial Clustering and labeling for mmWave Radar Point Cloud Data , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[21]  Xiangyu Gao,et al.  Experiments with mmWave Automotive Radar Test-bed , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[22]  Jürgen Dickmann,et al.  Extended Object Tracking assisted Adaptive Clustering for Radar in Autonomous Driving Applications , 2019, 2019 Sensor Data Fusion: Trends, Solutions, Applications (SDF).

[23]  Nils Appenrodt,et al.  A Multi-Stage Clustering Framework for Automotive Radar Data , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[24]  Christian Sturm,et al.  Semantic Segmentation on Automotive Radar Maps , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[25]  Klaus C. J. Dietmayer,et al.  2D Car Detection in Radar Data with PointNets , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[26]  Jürgen Dickmann,et al.  Semantic Segmentation on Radar Point Clouds , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[27]  Jürgen Dickmann,et al.  Radar-based Feature Design and Multiclass Classification for Road User Recognition , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[28]  Ulrich Neumann,et al.  SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Jürgen Dickmann,et al.  Comparison of random forest and long short-term memory network performances in classification tasks using radar , 2017, 2017 Sensor Data Fusion: Trends, Solutions, Applications (SDF).

[30]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[31]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[32]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Nils Appenrodt,et al.  "Automotive radar the key technology for autonomous driving: From detection and ranging to environmental understanding" , 2016, 2016 IEEE Radar Conference (RadarConf).

[34]  Thomas Wagner,et al.  Modification of DBSCAN and application to range/Doppler/DoA measurements for pedestrian recognition with an automotive radar system , 2015, 2015 European Radar Conference (EuRAD).

[35]  Derya Birant,et al.  ST-DBSCAN: An algorithm for clustering spatial-temporal data , 2007, Data Knowl. Eng..

[36]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.