A Three-Dimensional Deep Learning Framework for Human Behavior Analysis Using Range-Doppler Time Points

Deep neural networks have shown promise in the radar-based human activity analysis application. Different from existing deep learning models that take either micro-Doppler spectrograms or range profiles as their input, the proposed method can process micromotion signatures in a 3-D way. In this letter, we first transform radar echoes into range-Doppler (RD) time points and then directly process the point sets via a designed 3-D network called the RD PointNet. In fact, our point model is a discrete representation of the motion trajectory. Through this quantitative model, we can use the 3-D network to simultaneously capture human motion profiles and temporal variations. The motion capture simulations and ultrawideband radar measurements show that the proposed framework can achieve superior classification accuracy and noise robustness when compared with image-based methods.

[1]  Shobha Sundar Ram,et al.  Detection of Multiple Movers Based on Single Channel Source Separation of Their Micro-Dopplers , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Yuan He,et al.  Range-Doppler surface: a tool to analyse human target in ultra-wideband radar , 2015 .

[3]  Youngwook Kim,et al.  Micro-Doppler Based Classification of Human Aquatic Activities via Transfer Learning of Convolutional Neural Networks , 2016, Sensors.

[4]  Ivan Poupyrev,et al.  Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum , 2016, UIST.

[5]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Yuan He,et al.  Human Target Tracking in Multistatic Ultra-Wideband Radar , 2014 .

[7]  Gang Li,et al.  Micro-Doppler Parameter Estimation via Parametric Sparse Representation and Pruned Orthogonal Matching Pursuit , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Carmine Clemente,et al.  Developments in target micro-Doppler signatures analysis: radar imaging, ultrasound and through-the-wall radar , 2013, EURASIP J. Adv. Signal Process..

[9]  Yimin Zhang,et al.  Human motion recognition exploiting radar with stacked recurrent neural network , 2019, Digit. Signal Process..

[10]  Moeness G. Amin,et al.  Radar Data Cube Analysis for Fall Detection , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Youngwook Kim,et al.  Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[13]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[14]  Moeness Amin,et al.  Fall Detection Using Deep Learning in Range-Doppler Radars , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[15]  Ram M. Narayanan,et al.  Radar classification of indoor targets using support vector machines , 2016 .

[16]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[17]  Victor C. Chen,et al.  Analysis of radar micro-Doppler with time-frequency transform , 2000, Proceedings of the Tenth IEEE Workshop on Statistical Signal and Array Processing (Cat. No.00TH8496).

[18]  Hao Ling,et al.  Microdoppler signature simulation of computer animated human and animal motions , 2008, 2008 IEEE Antennas and Propagation Society International Symposium.

[19]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[20]  Melda Yuksel,et al.  Information-Theoretic Feature Selection for Human Micro-Doppler Signature Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[21]  S. Z. Gürbüz,et al.  Deep Neural Network Initialization Methods for Micro-Doppler Classification With Low Training Sample Support , 2017, IEEE Geoscience and Remote Sensing Letters.

[22]  Yazhou Wang,et al.  CW and Pulse–Doppler Radar Processing Based on FPGA for Human Sensing Applications , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Andreas G. Andreou,et al.  Action recognition using micro-Doppler signatures and a recurrent neural network , 2017, 2017 51st Annual Conference on Information Sciences and Systems (CISS).

[24]  Gang Li,et al.  Personnel Recognition and Gait Classification Based on Multistatic Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[25]  Mu Zhou,et al.  Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor , 2018, IEEE Sensors Journal.

[26]  Moeness G. Amin,et al.  Fall motion detection using combined range and Doppler features , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[27]  S. Z. Gürbüz,et al.  Deep convolutional autoencoder for radar-based classification of similar aided and unaided human activities , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[28]  Urs Schneider,et al.  Real-time capable micro-Doppler signature decomposition of walking human limbs , 2017, 2017 IEEE Radar Conference (RadarConf).