Recognition of hand gestures from cyclic hand movements using spatial-temporal features

Dynamic hand gesture recognition is a challenge field evenly this topic has been studied for a long time because of lack of feasible techniques deployed for Human-Computer Interaction (HCI) applications. In this paper, we propose a new type of gestures which presents a cyclic pattern of hand shapes during a movement. Through mapping of commands (e.g., turn devices on/off; increasing volume/channel) as output of a gesture recognition system, main purposes of the proposed gestures are to provide a natural and feasible way in control alliances in a smart home such as television, light, fan, door, so on. The proposed gestures are represented by both hand shapes and directions. Thanks to cyclic pattern of the hand shapes during performing a command, hand gestures are more easily segmented from video stream. We then focus on several challenges of the proposed gestures such as: non-synchronization phase of the gestures, change of hand shapes along temporal dimension and direction of hand movements. Such issues are addressed using combinations of spatial and temporal features extracted from consecutive frames of a gesture. The proposed algorithms are evaluated on several subjects. Evaluation results confirm that the proposed method obtains accuracy rates at 96% for segmenting a dynamic hand gesture and 95% for recognizing a command, averagely.

[1]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[2]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[3]  Antonis A. Argyros,et al.  Vision-based Hand Gesture Recognition for Human-Computer Interaction , 2008 .

[4]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  N.D. Georganas,et al.  A dynamic gesture interface for virtual environments based on hidden Markov models , 2005, IEEE International Workshop on Haptic Audio Visual Environments and their Applications.

[6]  Eric Castelli,et al.  Improvements of RGB-D hand posture recognition using an user-guide scheme , 2015, 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[7]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[8]  Z. Liu,et al.  A real time system for dynamic hand gesture recognition with a depth sensor , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[9]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Gang Hua,et al.  Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields , 2012, Image Vis. Comput..

[11]  Daijin Kim,et al.  Simultaneous Gesture Segmentation and Recognition based on Forward Spotting Accumulative HMMs , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[13]  Sergio Escalera,et al.  Multi-modal gesture recognition challenge 2013: dataset and results , 2013, ICMI '13.

[14]  Sébastien Marcel,et al.  Hand gesture recognition using input-output hidden Markov models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[15]  Chung-Lin Huang,et al.  A model-based hand gesture recognition system , 2001, Machine Vision and Applications.

[16]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[17]  Deyou Xu.  A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[18]  Josef Kittler,et al.  Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications , 2015 .

[19]  Juan Pablo Wachs,et al.  Hierarchical Elastic Graph Matching for Hand Gesture Recognition , 2012, CIARP.

[20]  Narciso García,et al.  Human-Computer Interaction based on visual recognition using volumegrams of Local Binary Patterns , 2015, 2015 IEEE International Conference on Consumer Electronics (ICCE).

[21]  Markus Koskela,et al.  Online RGB-D gesture recognition with extreme learning machines , 2013, ICMI '13.

[22]  Ming C. Leu,et al.  American Sign Language word recognition with a sensory glove using artificial neural networks , 2011, Eng. Appl. Artif. Intell..

[23]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Ayoub Al-Hamadi,et al.  Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences , 2008, J. WSCG.

[25]  R. Oka,et al.  Spotting Recognition of Human Gestures from Motion Images , 1994 .

[26]  Immanuel Bayer,et al.  A multi modal approach to gesture recognition from audio and video data , 2013, ICMI '13.