Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

[1]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[2]  Geoffrey E. Hinton,et al.  Glove-talk II - a neural-network interface which maps gestures to parallel formant speech synthesizer controls , 1997, IEEE Trans. Neural Networks.

[3]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[4]  Scott K. Liddell,et al.  American Sign Language: The Phonological Base , 2013 .

[5]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition , 2008 .

[6]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[8]  Cem Keskin,et al.  REAL TIME HAND TRACKING AND 3D GESTURE RECOGNITION FOR INTERACTIVE INTERFACES USING HMM , 2003 .

[9]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[11]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[12]  Michael J. Black,et al.  A Probabilistic Framework for Matching Temporal Trajectories: CONDENSATION-Based Recognition of Gestures and Expressions , 1998, ECCV.

[13]  Kwang-Hyun Park,et al.  Continuous gesture recognition system for Korean sign language based on fuzzy logic and hidden Markov model , 2002, 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems. FUZZ-IEEE'02. Proceedings (Cat. No.02CH37291).

[14]  Ayoub Al-Hamadi,et al.  Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences , 2008, J. WSCG.

[15]  Christian C. Wagner,et al.  Survey on classifying human actions through visual sensors , 2012, Artificial Intelligence Review.

[16]  Rini Akmeliawati,et al.  A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking , 2010, 2010 IEEE International Conference on Mechatronics and Automation.

[17]  Ming Ouhyoung,et al.  A sign language recognition system using hidden markov model and context sensitive search , 1996, VRST.

[18]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.

[19]  Wen Gao,et al.  A Chinese sign language recognition system based on SOFM/SRN/HMM , 2004, Pattern Recognit..

[20]  Annelies Braffort.  ARGo: An Architecture for Sign Language Recognition and Interpretation , 1996, Gesture Workshop.

[21]  Yoshiaki Shirai,et al.  Extraction of Hand Features for Recognition of Sign Language Words , 2002 .

[22]  Agnès Just,et al.  HMM and IOHMM for the Recognition of Mono- and Bi-Manual 3D Hand Gestures , 2004, BMVC.

[23]  Dimitris N. Metaxas,et al.  Parallel hidden Markov models for American sign language recognition , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Dimitris N. Metaxas,et al.  Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes , 1999, Gesture Workshop.

[25]  Hyung Lee-Kwang,et al.  Modeling and recognition of hand gesture using colored Petri nets , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[26]  Ying Wu,et al.  Hand modeling, analysis and recognition , 2001, IEEE Signal Process. Mag..

[27]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[29]  Wen Gao,et al.  Transition movement models for large vocabulary continuous sign language recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[30]  Gerhard Rigoll,et al.  New improved feature extraction methods for real-time high performance image sequence recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[31]  Pei Yin,et al.  American Sign Language Phrase Verification in an Educational Game for Deaf Children , 2010, 2010 20th International Conference on Pattern Recognition.

[32]  Nianjun Liu,et al.  Model structure selection & training algorithms for an HMM gesture recognition system , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[33]  Aaron F. Bobick,et al.  A State-Based Approach to the Representation and Recognition of Gesture , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Wen Gao,et al.  Recognition of sign language subwords based on boosted hidden Markov models , 2005, ICMI '05.

[35]  Bülent Sankur,et al.  SignTutor: An Interactive System for Sign Language Tutoring , 2009, IEEE Multimedia.

[36]  Munib Qutaishat,et al.  American sign language (ASL) recognition based on Hough transform and neural networks , 2007, Expert Syst. Appl..

[37]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[38]  Aditya Ramamoorthy,et al.  Recognition of dynamic hand gestures , 2003, Pattern Recognit..

[39]  Kirsti Grobel,et al.  Isolated sign language recognition using hidden Markov models , 1996, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[40]  Thad Starner,et al.  American sign language recognition in game development for deaf children , 2006, Assets '06.

[41]  Narendra Ahuja,et al.  Recognizing hand gesture using motion trajectories , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[42]  Khaled Assaleh,et al.  Video-based signer-independent Arabic sign language recognition using hidden Markov models , 2009, Appl. Soft Comput..

[43]  Kjersti Aas,et al.  Applications of hidden Markov chains in image analysis , 1999, Pattern Recognit..

[44]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[45]  Purnendu Acharjee,et al.  BODO Speech Recognition based on Hidden Markov Model Toolkit (HTK) , 2013 .

[46]  Iwan Njoto Sandjaja,et al.  Sign Language Number Recognition , 2009, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[47]  Alistair D. N. Edwards,et al.  Progress in Gestural Interaction, Proceedings of Gesture Workshop '96, March 19th 1996, University of York, UK , 1997, Gesture Workshop.

[48]  T. Kobayashi,et al.  Partly-hidden Markov model and its application to gesture recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[49]  Alexander Gruenstein.  Two Methods of Gesture Recognition , 2002 .

[50]  Wen Gao,et al.  A continuous Chinese sign language recognition system , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[51]  R. S. Jadon,et al.  A REVIEW OF VISION BASED HAND GESTURES RECOGNITION , 2009 .

[52]  Toshiaki Ejima,et al.  Real-Time Hand Tracking and Gesture Recognition System , 2005 .

[53]  Konstantinos G. P. Derpanis,et al.  A Review of Vision-Based Hand Gestures , 2004 .

[54]  Robyn A. Owens,et al.  Australian sign language recognition , 2005, Machine Vision and Applications.

[55]  Geoffrey E. Hinton,et al.  Glove-TalkII-a neural-network interface which maps gestures to parallel formant speech synthesizer controls , 1997, IEEE Trans. Neural Networks.

[56]  Karl-Friedrich Kraiss,et al.  Towards an Automatic Sign Language Recognition System Using Subunits , 2001, Gesture Workshop.

[57]  Gerhard Rigoll,et al.  High Performance Real-Time Gesture Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[58]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory , 2008, 2008 19th International Conference on Pattern Recognition.

[59]  Wen Gao,et al.  A novel approach to automatically extracting basic units from Chinese sign language , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[60]  Khaled Assaleh,et al.  Vision-based system for continuous Arabic Sign Language recognition in user dependent mode , 2008, 2008 5th International Symposium on Mechatronics and Its Applications.

[61]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[62]  Wen Gao,et al.  A SRN/HMM system for signer-independent continuous sign language recognition , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[63]  Zeung nam Bien,et al.  Continuous Korean Sign Language Recognition using Gesture Segmentation and Hidden Markov Model , 2002 .

[64]  Gerhard Rigoll,et al.  Hidden Markov model based continuous online gesture recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[65]  Wen Gao,et al.  Sign Language Recognition Based on HMM/ANN/DP , 2000, Int. J. Pattern Recognit. Artif. Intell..

[66]  Yasuo Horiuchi,et al.  Sign Language Recognition Based on Position and Movement Using Multi-Stream HMM , 2008, 2008 Second International Symposium on Universal Communication.

[67]  Wen Gao,et al.  A vision-based sign language recognition system using tied-mixture density HMM , 2004, ICMI '04.

[68]  Sanjeev Sofat,et al.  Vision Based Hand Gesture Recognition , 2009 .

[69]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..