A hand gesture recognition technique for human-computer interaction

Abstract We propose an approach to recognize trajectory-based dynamic hand gestures in real time for human–computer interaction (HCI). We also introduce a fast learning mechanism that does not require extensive training data to teach gestures to the system. We use a six-degrees-of-freedom position tracker to collect trajectory data and represent gestures as an ordered sequence of directional movements in 2D. In the learning phase, sample gesture data is filtered and processed to create gesture recognizers, which are basically finite-state machine sequence recognizers. We achieve online gesture recognition by these recognizers without needing to specify gesture start and end positions. The results of the conducted user study show that the proposed method is very promising in terms of gesture detection and recognition performance (73% accuracy) in a stream of motion. Additionally, the assessment of the user attitude survey denotes that the gestural interface is very useful and satisfactory. One of the novel parts of the proposed approach is that it gives users the freedom to create gesture commands according to their preferences for selected tasks. Thus, the presented gesture recognition approach makes the HCI process more intuitive and user specific.

[1]  Aaron F. Bobick,et al.  A State-Based Approach to the Representation and Recognition of Gesture , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Marco Roccetti,et al.  On the design and player satisfaction evaluation of an immersive gestural game: the case of tortellino x-perience at the Shanghai world expo , 2011, SIGDOC '11.

[3]  Wu-Chih Hu,et al.  Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination , 2011, Expert Syst. Appl..

[4]  Gang Ren,et al.  3D selection with freehand gesture , 2013, Comput. Graph..

[5]  Thomas S. Huang,et al.  Gesture modeling and recognition using finite state machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[6]  Aditya Ramamoorthy,et al.  Recognition of dynamic hand gestures , 2003, Pattern Recognit..

[7]  Mohammed Yeasin,et al.  Visual understanding of dynamic hand gestures , 2000, Pattern Recognit..

[8]  Ming C. Leu,et al.  Human-Computer Interaction System with Artificial Neural Network Using Motion Tracker and Data Glove , 2005, PReMI.

[9]  M. S. Sahane,et al.  Visual Interpretation Of Hand Gestures For Human Computer Interaction , 2014 .

[10]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[11]  Gerhard Rigoll,et al.  High Performance Real-Time Gesture Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[12]  Brian Shackel,et al.  Usability - Context, framework, definition, design and evaluation , 1991, Interact. Comput..

[13]  Biing-Hwang Juang,et al.  Feature Processing and Modeling for 6D Motion Gesture Recognition , 2013, IEEE Transactions on Multimedia.

[14]  Ralf Salomon,et al.  Gesture recognition for virtual reality applications using data gloves and neural networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[15]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.

[16]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Vasile Gui,et al.  Dynamic hand gesture recognition for human-computer interactions , 2011, 2011 6th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI).

[18]  Chen-Chiung Hsieh,et al.  A real time hand gesture recognition system using motion history image , 2010, 2010 2nd International Conference on Signal Processing Systems.

[19]  Aiguo Song,et al.  Dynamic hand gesture recognition based on SURF tracking , 2011, 2011 International Conference on Electric Information and Control Engineering.

[20]  Gerhard Rigoll,et al.  Hidden Markov model based continuous online gesture recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[21]  Yi Li,et al.  Dynamic hand gesture recognition using hidden Markov models , 2012, 2012 7th International Conference on Computer Science & Education (ICCSE).

[22]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[23]  Hui Li,et al.  Kinect Based Dynamic Hand Gesture Recognition Algorithm Research , 2012, 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics.

[24]  Christus,et al.  A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins , 2022 .

[25]  Xiaoyan Wang,et al.  Hidden-Markov-Models-Based Dynamic Hand Gesture Recognition , 2012 .

[26]  Donald A. Norman,et al.  Natural user interfaces are not natural , 2010, INTR.

[27]  Marco Roccetti,et al.  The art and craft of making the Tortellino: playing with a digital gesture recognizer for preparing pasta culinary recipes , 2010, CIE.

[28]  Bharti Bansal,et al.  Gesture Recognition: A Survey , 2016 .

[29]  S. Mohamed Mansoor Roomi,et al.  Hand Gesture Recognition for Human-Computer Interaction , 2010 .

[30]  Helge J. Ritter,et al.  Visual recognition of continuous hand postures , 2002, IEEE Trans. Neural Networks.

[31]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[32]  Marco Roccetti,et al.  Playing into the wild: A gesture-based interface for gaming in public spaces , 2012, J. Vis. Commun. Image Represent..

[33]  Gang Hua,et al.  Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields , 2012, Image Vis. Comput..