Human Motion Serialization Recognition With Through-the-Wall Radar

Motivated by the intrinsic dynamics of physical motion as well as establishment of target motion model, this article addresses the problem of human motion recognition with ultra wide band (UWB) through-the-wall radar (TWR) in a novel view of range profile serialization. Specifically, we first convert the original radar echoes into range profiles. Then, an auto-encoder network (AEN) with three dense layers is adopted to reduce the dimension and extract the features of each range profile. After that, a gated recurrent unit (GRU) network with two hidden layers is employed to deal with the features of each time-range slice and output the recognition results at each slice in real time. Finally, experimental data with respect to four different behind-wall human motions is collected by self-developed UWB TWR to validate the effectiveness of the proposed model. The results show that the proposed model can validly recognize the human motion serialization and achieve 93% recognition accuracy within the initial 20% duration of the activities (the average durations are 4s, 5.5s, 3s and 4.5s), which is of great significance for real-time human motion recognition.

[1]  Youngwook Kim,et al.  Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Hon Keung Kwan,et al.  Recurrent neural network design for temporal sequence learning , 2000, Proceedings of the 43rd IEEE Midwest Symposium on Circuits and Systems (Cat.No.CH37144).

[3]  Jean-Yves Tourneret,et al.  Clutter rejection for MTI radar using a single antenna and a long integration time , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[4]  Xinyu Li,et al.  A Survey of Deep Learning-Based Human Activity Recognition in Radar , 2019, Remote. Sens..

[5]  Jinping Sun,et al.  Parameter estimation method of walking human based on radar micro-Doppler , 2017, 2017 IEEE Radar Conference (RadarConf).

[6]  Siyang Cao,et al.  Real-Time Human Motion Behavior Detection via CNN Using mmWave Radar , 2019, IEEE Sensors Letters.

[7]  Lihui Wang,et al.  Deep learning-based human motion recognition for predictive context-aware human-robot collaboration , 2018 .

[8]  Haiquan Chen,et al.  A Hybrid CNN–LSTM Network for the Classification of Human Activities Based on Micro-Doppler Radar , 2020, IEEE Access.

[9]  Zhao Li,et al.  MHHT-Based Method for Analysis of Micro-Doppler Signatures for Human Finer-Grained Activity Using Through-Wall SFCW Radar , 2017, Remote. Sens..

[10]  Yimin Zhang,et al.  Human motion recognition exploiting radar with stacked recurrent neural network , 2019, Digit. Signal Process..

[11]  Lingjiang Kong,et al.  An Imaging Dictionary Based Multipath Suppression Algorithm for Through-Wall Radar Imaging , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[12]  Yimin Zhang,et al.  Radar Signal Processing for Elderly Fall Detection: The future for in-home monitoring , 2016, IEEE Signal Processing Magazine.

[13]  Duo Zhang,et al.  Use Long Short-Term Memory to Enhance Internet of Things for Combined Sewer Overflow Monitoring , 2018 .

[14]  Kejun Wang,et al.  Video-Based Abnormal Human Behavior Recognition—A Review , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[15]  Branka Jokanovic,et al.  Radar fall motion detection using deep learning , 2016, 2016 IEEE Radar Conference (RadarConf).

[16]  Francesco Fioranelli,et al.  Feature Diversity for Optimized Human Micro-Doppler Classification Using Multistatic Radar , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[17]  Jin Young Kim,et al.  Classification of Human Postures Using Ultra-Wide Band Radar Based on Neural Networks , 2014, 2014 International Conference on IT Convergence and Security (ICITCS).

[18]  Lingjiang Kong,et al.  Robust Human Targets Tracking for MIMO Through-Wall Radar via Multi-Algorithm Fusion , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Yuan He,et al.  Segmented convolutional gated recurrent neural networks for human activity recognition in ultra-wideband radar , 2020, Neurocomputing.

[20]  Hans Driessen,et al.  Human motion classification using a particle filter approach: Multiple model particle filtering applied to the micro-Doppler spectrum , 2013 .

[21]  Ahmet Arslan,et al.  Through-Wall Radar Classification of Human Posture Using Convolutional Neural Networks , 2019, International Journal of Antennas and Propagation.

[22]  Eiji Oki,et al.  ECG Monitoring System Integrated With IR-UWB Radar Based on CNN , 2016, IEEE Access.

[23]  Lingjiang Kong,et al.  Reconstruction Filter Design for Stepped-Frequency Continuous Wave , 2012, IEEE Transactions on Signal Processing.

[24]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[25]  Martin A. Riedmiller,et al.  Deep auto-encoder neural networks in reinforcement learning , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[26]  Patrick van der Smagt,et al.  Two-stream RNN/CNN for action recognition in 3D videos , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Youngwook Kim,et al.  Human Activity Classification With Transmission and Reflection Coefficients of On-Body Antennas Through Deep Convolutional Neural Networks , 2017, IEEE Transactions on Antennas and Propagation.

[28]  Lingjiang Kong,et al.  Through-Wall Human Motion Representation via Autoencoder-Self Organized Mapping Network , 2019, 2019 IEEE Radar Conference (RadarConf).

[29]  Abdesselam Bouzerdoum,et al.  Human Motion Classification with Micro-Doppler Radar and Bayesian-Optimized Convolutional Neural Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[31]  Baoju Zhang,et al.  Experimental study of through-wall human detection using ultra wideband radar sensors , 2014 .

[32]  Xiaohua Zhu,et al.  Non-Contact Human Motion Recognition Based on UWB Radar , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[33]  Olivier Romain,et al.  Performances of Multitones for Ultra-Wideband Software-Defined Radar , 2017, IEEE Access.

[34]  Bahri Cagliyan,et al.  Micro-Doppler-Based Human Activity Classification Using the Mote-Scale BumbleBee Radar , 2015, IEEE Geoscience and Remote Sensing Letters.

[35]  Youngwook Kim,et al.  Classification of human activities on UWB radar using a support vector machine , 2010, 2010 IEEE Antennas and Propagation Society International Symposium.

[36]  Francesco Fioranelli,et al.  Bi-LSTM Network for Multimodal Continuous Human Activity Recognition and Fall Detection , 2020, IEEE Sensors Journal.