Dynamic gesture recognition using transformation invariant hand shape recognition

In this thesis a detailed framework is presented for accurate real time gesture recognition. Our approach to develop a hand-shape classifier, trained using computer animation, along with its application in dynamic gesture recognition is described. The system developed operates in real time and provides accurate gesture recognition. It operates using a single low resolution camera and operates in Matlab on a conventional PC running Windows XP. The hand shape classifier outlined in this thesis uses transformation invariant subspaces created using Principal Component Analysis (PCA). These subspaces are created from a large vocabulary created in a systematic maimer using computer animation. In recognising dynamic gestures we utilise both hand shape and hand position information; these are two o f the main features used by humans in distinguishing gestures. Hidden Markov Models (HMMs) are trained and employed to recognise this combination of hand shape and hand position features. During the course o f this thesis we have described in detail the inspiration and motivation behind our research and its possible applications. In this work our emphasis is on achieving a high speed system that works in real time with high accuracy.

[1]  Atid Shamaie.  Hand tracking and bimanual movement understanding , 2003 .

[2]  Paulo Peixoto,et al.  A Vision Based Interface for Local Collaborative Music Synthesis , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[3]  C Baur,et al.  A non-contact mouse for surgeon-computer interaction. , 2004, Technology and health care : official journal of the European Society for Engineering and Medicine.

[4]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[5]  Horst Bischof,et al.  Multiple eigenspaces , 2002, Pattern Recognit..

[6]  Sumantra Dutta Roy,et al.  Hand gesture modelling and recognition involving changing shapes and trajectories, using a Predictive EigenTracker , 2007, Pattern Recognit. Lett..

[7]  Agnès Just,et al.  Hand Posture Classification and Recognition using the Modified Census Transform , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[8]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Lalit Gupta,et al.  Gesture-based interaction and communication: automated classification of hand gesture contours , 2001, IEEE Trans. Syst. Man Cybern. Syst..

[10]  James W. Davis,et al.  Real-time recognition of activity using temporal templates , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[11]  Mingjing Li,et al.  Multi-view face detection with FloatBoost , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[12]  Wen Yi Zhao.  Improving the robustness of subspace FR system , 1999 .

[13]  George Awad,et al.  Real Time Hand Gesture Recognition Including Hand Segmentation and Tracking , 2006, ISVC.

[14]  Richard Bowden,et al.  A boosted classifier tree for hand shape detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[15]  Jochen Triesch,et al.  Classification of hand postures against complex backgrounds using elastic graph matching , 2002, Image Vis. Comput..

[16]  Qi Tian,et al.  Local Motion Analysis and Its Application in Video based Swimming Style Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[17]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[19]  Andrew Zisserman,et al.  Minimal Training, Large Lexicon, Unconstrained Sign Language Recognition , 2004, BMVC.

[20]  José Miguel Salles Dias,et al.  O.G.R.E. - Open Gestures Recognition Engine , 2004, SIBGRAPI.

[21]  Markus Kohler.  System architecture and techniques for gesture recognition in unconstrained environments , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[22]  Ying Wu,et al.  View-independent recognition of hand postures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[23]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[24]  R. J. KohlerInformatik.  System Architecture and Techniques for Gesture Recognition in Unconstraint Environments , 1997 .

[25]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[26]  Surendra Ranganath,et al.  Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  K. Barner,et al.  An Active Shape Model Based Tactile Hand Shape Recognition with Support Vector Machines , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[28]  Zoubin Ghahramani,et al.  An Introduction to Hidden Markov Models and Bayesian Networks , 2001, Int. J. Pattern Recognit. Artif. Intell..

[29]  Horst Bunke,et al.  Hidden Markov models: applications in computer vision , 2001 .

[30]  Yann LeCun,et al.  Transformation Invariance in Pattern Recognition-Tangent Distance and Tangent Propagation , 1996, Neural Networks: Tricks of the Trade.

[31]  Alistair Sutherland,et al.  Accurate Recognition of Large Number of Hand Gestures , 2003 .

[32]  Gang Qian,et al.  Dance posture recognition using wide-baseline orthogonal stereo cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[33]  Rongchun Zhao,et al.  Automatic Gait Recognition using Dynamic Variance Features , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).