Human motion recognition exploiting radar with stacked recurrent neural network

Abstract We develop a novel radar-based human motion recognition technique that exploits the temporal sequentiality of human motions. The stacked recurrent neural network (RNN) with long short-term memory (LSTM) units is employed to extract sequential features for automatic motion classification. The spectrogram of raw radar data is used as the network input to utilize the time-varying Doppler and micro-Doppler signatures for human motion characterization. Based on experimental data, we verified that a stacked RNN with two 36-cell LSTM layers successfully classifies six different types of human motions.

[1]  Daniel Thalmann,et al.  A global human walking model with real-time kinematic personification , 1990, The Visual Computer.

[2]  Giovanni Saggio,et al.  Modeling Wearable Bend Sensor Behavior for Human Motion Capture , 2014, IEEE Sensors Journal.

[3]  Moeness G. Amin,et al.  Automatic Data-Driven Frequency-Warped Cepstral Feature Design for Micro-Doppler Classification , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[4]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[5]  Marek R. Ogiela,et al.  Full body movements recognition - unsupervised learning approach with heuristic R-GDL method , 2015, Digit. Signal Process..

[6]  Tao Zhang,et al.  An Adaptive S-Method to Analyze Micro-Doppler Signals for Human Activity Classification , 2017, Sensors.

[7]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[8]  Yang Yang,et al.  One-shot learning based pattern transition map for action early recognition , 2018, Signal Process..

[9]  Ram M. Narayanan,et al.  Radar micro-Doppler based human activity classification for indoor and outdoor environments , 2016, SPIE Defense + Security.

[10]  Ugur Güdükbay,et al.  Motion capture and human pose reconstruction from a single-view video sequence , 2013, Digit. Signal Process..

[11]  Yimin Zhang,et al.  Radar Signal Processing for Elderly Fall Detection: The future for in-home monitoring , 2016, IEEE Signal Processing Magazine.

[12]  Alessio Balleri,et al.  Recognition of humans based on radar micro-Doppler shape spectrum features , 2015 .

[13]  Wenbing Zhao,et al.  A Survey of Applications and Human Motion Recognition with Microsoft Kinect , 2015, Int. J. Pattern Recognit. Artif. Intell..

[14]  Yang Yang,et al.  Recurrent attention network using spatial-temporal relations for action recognition , 2018, Signal Process..

[15]  H. Wechsler,et al.  Micro-Doppler effect in radar: phenomenon, model, and simulation study , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[16]  Hongxun Yao,et al.  Distinctive action sketch for human action recognition , 2018, Signal Process..

[17]  Ah Chung Tsoi,et al.  Investigating the impact of frame rate towards robust human action recognition , 2016, Signal Process..

[18]  Branka Jokanovic,et al.  Multi-window time–frequency signature reconstruction from undersampled continuous-wave radar measurements for fall detection , 2015 .

[19]  S. Z. Gürbüz,et al.  Deep Neural Network Initialization Methods for Micro-Doppler Classification With Low Training Sample Support , 2017, IEEE Geoscience and Remote Sensing Letters.

[20]  Abdesselam Bouzerdoum,et al.  A Human Gait Classification Method Based on Radar Doppler Spectrograms , 2010, EURASIP J. Adv. Signal Process..

[21]  Dapeng Tao,et al.  Skeleton embedded motion body partition for human action recognition using depth sequences , 2018, Signal Process..

[22]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[23]  Carmine Clemente,et al.  Micro-doppler-based in-home aided and unaided walking recognition with multiple radar and sonar systems , 2017 .

[24]  Boualem Boashash,et al.  Time-Frequency Signal Analysis and Processing: A Comprehensive Reference , 2015 .

[25]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[26]  Wenbing Tao,et al.  Radar-based fall detection based on Doppler time-frequency signatures for assisted living , 2015 .

[27]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[28]  Lingjiang Kong,et al.  Human body and limb motion recognition via stacked gated recurrent units network , 2018, IET Radar, Sonar & Navigation.

[29]  Branka Jokanovic,et al.  Multiple joint-variable domains recognition of human motion , 2017, 2017 IEEE Radar Conference (RadarConf).

[30]  Gang Li,et al.  Personnel Recognition and Gait Classification Based on Multistatic Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[31]  Birsen Yazici,et al.  Deep learning for radar , 2017, 2017 IEEE Radar Conference (RadarConf).