Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural Network

In this paper, we investigate the feasibility of recognizing human hand gestures using micro-Doppler signatures measured by Doppler radar with a deep convolutional neural network (DCNN). Hand gesture recognition using radar can be applied to control electronic appliances. Compared with an optical recognition system, radar can work regardless of light conditions and it can be embedded in a case. We classify ten different hand gestures, with only micro-Doppler signatures on spectrograms without range information. The ten gestures, which included swiping from left to right, swiping from right to left, rotating clockwise, rotating counterclockwise, pushing, double pushing, holding, and double holding, were measured using Doppler radar and their spectrograms investigated. A DCNN was employed to classify the spectrograms, with 90% of the data utilized for training and the remaining 10% for validation. After five-fold validation, the classification accuracy of the proposed method was found to be 85.6%. With seven gestures, the accuracy increased to 93.1%.

[1]  Ram M. Narayanan,et al.  Multistatic micro-doppler radar for determining target orientation and activity classification , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Dave Tahmoush,et al.  Radar micro-doppler for long range front-view gait recognition , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[3]  Youngwook Kim,et al.  Detection of Eye Blinking Using Doppler Sensor With Principal Component Analysis , 2015, IEEE Antennas and Wireless Propagation Letters.

[4]  Junsong Yuan,et al.  Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction , 2011, 2011 8th International Conference on Information, Communications & Signal Processing.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Pavlo Molchanov,et al.  Short-range FMCW monopulse radar for hand-gesture sensing , 2015, 2015 IEEE Radar Conference (RadarCon).

[7]  Ho-Sub Yoon,et al.  Hand gesture recognition using hidden Markov models , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[8]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[9]  Youngwook Kim,et al.  Classification of human activity on water through micro-Dopplers using deep convolutional neural networks , 2016, SPIE Defense + Security.

[10]  Christian Waldschmidt,et al.  RCS measurements of a human hand for radar-based gesture recognition at E-band , 2016, 2016 German Microwave Conference (GeMiC).

[11]  Francesco Fioranelli,et al.  Aspect angle dependence and multistatic data fusion for micro-Doppler classification of armed/unarmed personnel , 2015 .

[12]  Youngwook Kim,et al.  Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[14]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[15]  Ram M. Narayanan,et al.  Classification of human motions using empirical mode decomposition of human micro-Doppler signatures , 2014 .

[16]  Desney S. Tan,et al.  SoundWave: using the doppler effect to sense gestures , 2012, CHI.

[17]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[18]  Yi Yao,et al.  Hand gesture recognition and spotting in uncontrolled environments based on classifier weighting , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[19]  Youngwook Kim,et al.  Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[20]  Pavlo Molchanov,et al.  Multi-sensor system for driver's hand-gesture recognition , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[21]  Youngwook Kim,et al.  Application of Linear Predictive Coding for Human Activity Classification Based on Micro-Doppler Signatures , 2014, IEEE Geoscience and Remote Sensing Letters.

[22]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..