Practical classification of different moving targets using automotive radar and deep neural networks

In this work, the authors present results for classification of different classes of targets (car, single and multiple people, bicycle) using automotive radar data and different neural networks. A fast implementation of radar algorithms for detection, tracking, and micro-Doppler extraction is proposed in conjunction with the automotive radar transceiver TEF810X and microcontroller unit SR32R274 manufactured by NXP Semiconductors. Three different types of neural networks are considered, namely a classic convolutional network, a residual network, and a combination of convolutional and recurrent network, for different classification problems across the four classes of targets recorded. Considerable accuracy (close to 100% in some cases) and low latency of the radar pre-processing prior to classification (∼0.55 s to produce a 0.5 s long spectrogram) are demonstrated in this study, and possible shortcomings and outstanding issues are discussed.

[1]  L. Cifola,et al.  Multi-target human gait classification using LSTM recurrent neural networks applied to micro-Doppler , 2017, 2017 European Radar Conference (EURAD).

[2]  Shobha Sundar Ram,et al.  Dictionary learning for classification of indoor micro-Doppler signatures across multiple carriers , 2017, 2017 IEEE Radar Conference (RadarConf).

[3]  Wolfgang Menzel,et al.  Clustering of high resolution automotive radar detections and subsequent feature extraction for classification of road users , 2015, 2015 16th International Radar Symposium (IRS).

[4]  Francesco Fioranelli,et al.  Multistatic radar classification of armed vs unarmed personnel using neural networks , 2018, Evol. Syst..

[5]  Moeness Amin,et al.  Fall Detection Using Deep Learning in Range-Doppler Radars , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[6]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[7]  Branka Jokanovic,et al.  Effect of data representations on deep learning in fall detection , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[8]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[9]  Chi-Chih Chen,et al.  Micro-Doppler characteristics of pedestrians and bicycles for automotive radar sensors at 77 GHz , 2017, 2017 11th European Conference on Antennas and Propagation (EUCAP).

[10]  Hermann Rohling,et al.  Two-stage pedestrian classification in automotive radar systems , 2011, 2011 12th International Radar Symposium (IRS).

[11]  S. Z. Gürbüz,et al.  Deep Neural Network Initialization Methods for Micro-Doppler Classification With Low Training Sample Support , 2017, IEEE Geoscience and Remote Sensing Letters.

[12]  Dave Tahmoush,et al.  Review of micro-Doppler signatures , 2015 .

[13]  Branka Jokanovic,et al.  Multiple joint-variable domains recognition of human motion , 2017, 2017 IEEE Radar Conference (RadarConf).

[14]  Andy Stove,et al.  Predicted sensitivity of a 300GHz FMCW radar to pedestrians , 2016, 2016 46th European Microwave Conference (EuMC).

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  André Bourdoux,et al.  Micro-Doppler feature extraction using convolutional auto-encoders for low latency target classification , 2017, 2017 IEEE Radar Conference (RadarConf).

[17]  L. Cifola,et al.  Multi-target human gait classification using deep convolutional neural networks on micro-doppler spectrograms , 2016, 2016 European Radar Conference (EuRAD).

[18]  Jurgen Hasch,et al.  Driving towards 2020: Automotive radar technology trends , 2015, 2015 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[19]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[20]  J. Munkres ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[21]  S. Z. Gürbüz,et al.  Deep learning of micro-Doppler features for aided and unaided gait recognition , 2017, 2017 IEEE Radar Conference (RadarConf).

[22]  Seong-Cheol Kim,et al.  Human–vehicle classification using feature-based SVM in 77-GHz automotive FMCW radar , 2017 .

[23]  M. Gashinova,et al.  Comparison of pedestrian reflectivities at 24 and 300 GHz , 2017, 2017 18th International Radar Symposium (IRS).

[24]  M. Gashinova,et al.  Radar reflectivity and motion characteristics of pedestrians at 300 GHz , 2017, 2017 European Radar Conference (EURAD).

[25]  Wolfgang Menzel,et al.  Multi-target reflection point model of cyclists for automotive radar , 2017, 2017 European Radar Conference (EURAD).

[26]  Hermann Rohling,et al.  Radar CFAR Thresholding in Clutter and Multiple Target Situations , 1983, IEEE Transactions on Aerospace and Electronic Systems.

[27]  Younglok Kim,et al.  Target classification scheme using phase characteristics for automotive FMCW radar , 2016 .

[28]  Hermann Rohling,et al.  Pedestrian classification with 24 GHz chirp sequence radar , 2015, 2015 16th International Radar Symposium (IRS).

[29]  J. Khashbat,et al.  Determining the driver's reaction time in the stationary and real-life environments (Comparative Study) , 2012, 2012 7th International Forum on Strategic Technology (IFOST).

[30]  A. Rydberg,et al.  Harmonic automotive radar for VRU classification , 2009, 2009 International Radar Conference "Surveillance for a Safer World" (RADAR 2009).

[31]  Chi-Chih Chen,et al.  Bicycles and human riders backscattering at 77 GHz for automotive radar , 2016, 2016 10th European Conference on Antennas and Propagation (EuCAP).

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Francesco Fioranelli,et al.  Feature Diversity for Optimized Human Micro-Doppler Classification Using Multistatic Radar , 2017, IEEE Transactions on Aerospace and Electronic Systems.