Hand posture and gesture recognition techniques for virtual reality applications: a survey

Motion recognition is a topic in software engineering and dialect innovation with a goal of interpreting human signals through mathematical algorithm. Hand gesture is a strategy for nonverbal communication for individuals as it expresses more liberally than body parts. Hand gesture acknowledgment has more prominent significance in planning a proficient human computer interaction framework, utilizing signals as a characteristic interface favorable to circumstance of movements. Regardless, the distinguishing proof and acknowledgment of posture, gait, proxemics and human behaviors is furthermore the subject of motion to appreciate human nonverbal communication, thus building a richer bridge between machines and humans than primitive text user interfaces or even graphical user interfaces, which still limits the majority of input to electronics gadget. In this paper, a study on various motion recognition methodologies is given specific accentuation on available motions. A survey on hand posture and gesture is clarified with a detailed comparative analysis of hidden Markov model approach with other classifier techniques. Difficulties and future investigation bearing are also examined.

[1]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[2]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[3]  Xingyan Li.  Gesture Recognition Based on Fuzzy C-Means Clustering Algorithm , .

[4]  Jake K. Aggarwal,et al.  Articulated and elastic non-rigid motion: a review , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[5]  Xindong Wu,et al.  RIEVL: Recursive Induction Learning in Hand Gesture Recognition , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Javaid Iqbal,et al.  On the Improvement of Multi-Legged Locomotion over Difficult Terrains Using a Balance Stabilization Method: , 2012 .

[7]  Jerald Siby,et al.  Hand Gesture Recognition , 2022, SSRN Electronic Journal.

[8]  B. Prabhakaran,et al.  Segmentation and recognition of multi-attribute motion sequences , 2004, MULTIMEDIA '04.

[9]  Chuan-Jun Su,et al.  Kinect-enabled home-based rehabilitation system using Dynamic Time Warping and fuzzy logic , 2014, Appl. Soft Comput..

[10]  Ana González-Marcos,et al.  Biometric Identification through Hand Geometry Measurements , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Zhang Peng,et al.  An Automatic Hand Gesture Recognition System Based on Viola-Jones Method and SVMs , 2009, 2009 Second International Workshop on Computer Science and Engineering.

[12]  Jesús B. Alonso,et al.  Hand shape identification on multirange images , 2014, Inf. Sci..

[13]  Anil K. Jain,et al.  Deformable matching of hand shapes for user verification , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[14]  Weihua Sheng,et al.  Online hand gesture recognition using neural network based segmentation , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[16]  Thomas B. Moeslund,et al.  A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI , 2003, Gesture Workshop.

[17]  A. Al-Hamadi,et al.  Gesture Recognition for Alphabets from Hand Motion Trajectory Using Hidden Markov Models , 2007, 2007 IEEE International Symposium on Signal Processing and Information Technology.

[18]  Mohammed Yeasin,et al.  Visual understanding of dynamic hand gestures , 2000, Pattern Recognit..

[19]  Björn Stenger,et al.  Hand Pose Estimation Using Hierarchical Detection , 2004, ECCV Workshop on HCI.

[20]  Daijin Kim,et al.  Simultaneous Gesture Segmentation and Recognition based on Forward Spotting Accumulative HMMs , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[21]  Amit Gupta,et al.  FPGA based Real Time Human Hand Gesture Recognition System , 2012 .

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  Sungyoung Lee,et al.  Two-stage Hidden Markov Model in Gesture Recognition for Human Robot Interaction , 2012 .

[24]  Akira Iwata,et al.  A rotation invariant approach on static-gesture recognition using boundary histograms and neural networks , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[25]  Aditya Ramamoorthy,et al.  Recognition of dynamic hand gestures , 2003, Pattern Recognit..

[26]  Mathias Kölsch,et al.  Robust hand detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[27]  Geoffrey E. Hinton,et al.  Glove-TalkII-a neural-network interface which maps gestures to parallel formant speech synthesizer controls , 1997, IEEE Trans. Neural Networks.

[28]  Sébastien Marcel,et al.  Hand Posture Recognition in a Body-Face Centered Space , 1999, Gesture Workshop.

[29]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[30]  Nikos Papamarkos,et al.  Hand gesture recognition using a neural network shape fitting technique , 2009, Eng. Appl. Artif. Intell..

[31]  David B. Koons,et al.  Iconic: speech and depictive gestures at the human-machine interface , 1994, CHI Conference Companion.

[32]  Sanjay Meena,et al.  A Study on Hand Gesture Recognition Technique , 2011 .

[33]  Lars Bretzner,et al.  Using marking menus to develop command sets for computer vision based hand gesture interfaces , 2002, NordiCHI '02.

[34]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[35]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[36]  Koon Yin C. Chung.  Facial Expression Recognition: by Using Class Mean Gabor Responses with Kernel Principal Component Analysis , 2010 .

[37]  Alex Pentland,et al.  Looking at People: Sensing for Ubiquitous and Wearable Computing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Bin Yu,et al.  Feature learning based on SAE-PCA network for human gesture recognition in RGBD images , 2015, Neurocomputing.

[39]  Juan Pablo Wachs,et al.  HEGM: A hierarchical elastic graph matching for hand gesture recognition , 2014, Pattern Recognit..

[40]  Basilio Sierra,et al.  Layered architecture for real time sign recognition: Hand gesture and movement , 2010, Eng. Appl. Artif. Intell..

[41]  Bian Wu,et al.  A hand gesture recognition system based on local linear embedding , 2005, J. Vis. Lang. Comput..

[42]  Ming Xie,et al.  Estimation of the fundamental matrix from uncalibrated stereo hand images for 3D hand gesture recognition , 2003, Pattern Recognit..

[43]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Tin Hninn Hninn Maung,et al.  Real-Time Hand Tracking and Gesture Recognition System Using Neural Networks , 2009 .

[45]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[46]  Hanqing Lu,et al.  A real-time hand gesture recognition method , 2007, 2011 International Conference on Electronics, Communications and Control (ICECC).

[47]  Honghai Liu,et al.  Advances in View-Invariant Human Motion Analysis: A Review , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[48]  Ngọc Trần Nguyên,et al.  Real-time hand gesture recognition , 2013 .

[49]  Alan Wexelblat,et al.  An approach to natural gesture in virtual environments , 1995, TCHI.

[50]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[51]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[52]  Alessio Del Bue,et al.  Human behavior analysis in video surveillance: A Social Signal Processing perspective , 2013, Neurocomputing.

[53]  Yoichi Sato,et al.  Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[54]  Shuzhi Sam Ge,et al.  Hand Gesture Recognition and Tracking based on Distributed Locally Linear Embedding , 2006 .

[55]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[56]  Ekta Vats,et al.  Fuzzy human motion analysis: A review , 2014, Pattern Recognit..

[57]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[58]  Tamás Szirányi,et al.  User-adaptive hand gesture recognition system with interactive training , 2005, Image Vis. Comput..

[59]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[60]  Jochen Triesch,et al.  A System for Person-Independent Hand Posture Recognition against Complex Backgrounds , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  B. B. Zaidan,et al.  Image skin segmentation based on multi-agent learning Bayesian and neural network , 2014, Eng. Appl. Artif. Intell..

[62]  Zhishen Huang,et al.  Study of sign language recognition based on Gabor wavelet transforms , 2010, 2010 International Conference On Computer Design and Applications.

[63]  Yu Ren,et al.  Real-Time Hand Gesture Recognition Based on Vision , 2010, Edutainment.

[64]  Hongzhou Zhang,et al.  2D Gaborface representation method for face recognition with ensemble and multichannel model , 2008, Image Vis. Comput..

[65]  Yiping Yang,et al.  Real-time Hand Gesture Recognition from Depth Images Using Convex Shape Decomposition Method , 2014, J. Signal Process. Syst..

[66]  Antonio Fernández-Caballero,et al.  A survey of video datasets for human action and activity recognition , 2013, Comput. Vis. Image Underst..

[67]  Ai Poh Loh,et al.  Graph matching based hand posture recognition using neuro-biologically inspired features , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[68]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[69]  M. Maraqa,et al.  Recognition of Arabic Sign Language (ArSL) using recurrent neural networks , 2008, 2008 First International Conference on the Applications of Digital Information and Web Technologies (ICADIWT).

[70]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[71]  Kanad K. Biswas,et al.  Real Time Hand Tracking and Gesture Recognition , 2009, IPCV.

[72]  Rashid Ansari,et al.  Multimodal human discourse: gesture and speech , 2002, TCHI.

[73]  Doug A. Bowman.  Principles for the design of performance-oriented interaction techniques , 2002 .

[74]  Alan Liu,et al.  A Survey of Surgical Simulation: Applications, Technology, and Education , 2003, Presence: Teleoperators & Virtual Environments.

[75]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Richard Bowden,et al.  A boosted classifier tree for hand shape detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[77]  Ugur Güdükbay,et al.  A hand gesture recognition technique for human-computer interaction , 2015, J. Vis. Commun. Image Represent..

[78]  Sharath Pankanti,et al.  A Prototype Hand Geometry-based Verication System , 1999 .

[79]  Mohan M. Trivedi,et al.  Human action recognition using multiple views: a comparative perspective on recent developments , 2011, J-HGBU '11.

[80]  M. S. Sahane,et al.  Visual Interpretation Of Hand Gestures For Human Computer Interaction , 2014 .

[81]  Shahzad Malik,et al.  Visual touchpad: a two-handed gestural input device , 2004, ICMI '04.

[82]  Jianfeng Huang,et al.  Hand Gesture Recognition Using Appearance Features Based on 3D Point Cloud , 2016 .

[83]  Julien Letessier,et al.  Visual tracking of bare fingers for interactive surfaces , 2004, UIST '04.

[84]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[85]  Margrit Betke,et al.  Comparing random forest approaches to segmenting and classifying gestures , 2017, Image Vis. Comput..

[86]  Guodong Guo,et al.  A survey on still image based human action recognition , 2014, Pattern Recognit..

[87]  Wu-Chih Hu,et al.  Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination , 2011, Expert Syst. Appl..

[88]  Joseph L. Gabbard.  A Taxonomy of Usability Characteristics in Virtual Environments , 1997 .

[89]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[90]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[91]  Dmitry B. Goldgof,et al.  Gesture recognition using Bezier curves for visualization navigation from registered 3-D data , 2004, Pattern Recognit..

[92]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[93]  Yi Li,et al.  Features extraction from hand images based on new detection operators , 2011, Pattern Recognit..

[94]  Heung-Il Suk,et al.  Hand gesture recognition based on dynamic Bayesian network framework , 2010, Pattern Recognit..

[95]  Tsukasa Ogasawara,et al.  A hand-pose estimation for vision-based human interfaces , 2003, IEEE Trans. Ind. Electron..

[96]  Ganesh K. Venayagamoorthy,et al.  Recognition of facial expressions using Gabor wavelets and learning vector quantization , 2008, Eng. Appl. Artif. Intell..

[97]  Agnès Just,et al.  A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition , 2009, Comput. Vis. Image Underst..

[98]  Y. Lay,et al.  Hand shape recognition , 2000 .

[99]  Aytül Erçil,et al.  Combining implicit polynomials and geometric features for hand recognition , 2003, Pattern Recognit. Lett..

[100]  Dnyaneshwar Jadhav,et al.  Dynamic Gesture Recognition Using Hidden Markov Model in Static Background , 2011 .

[101]  David C. Hogg,et al.  Towards 3D hand tracking using a deformable model , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[102]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[103]  Madhuri S. Mulekar.  Data Mining: Multimedia, Soft Computing, and Bioinformatics , 2004, Technometrics.

[104]  Surendra Ranganath,et al.  Real-time gesture recognition system and application , 2002, Image Vis. Comput..

[105]  Mark Billinghurst,et al.  Put that where? voice and gesture at the graphics interface , 1998, COMG.

[106]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[107]  Yangsheng Xu,et al.  A realtime hand gesture recognition based on Haar wavelet representation , 2009, 2008 IEEE International Conference on Robotics and Biomimetics.

[108]  Andy Cockburn,et al.  FingARtips: gesture based direct manipulation in Augmented Reality , 2004, GRAPHITE '04.

[109]  Martin Saerbeck,et al.  Recent methods and databases in vision-based hand gesture recognition: A review , 2015, Comput. Vis. Image Underst..

[110]  Prahlad Vadakkepat,et al.  HAND POSTURE AND FACE RECOGNITION USING A FUZZY-ROUGH APPROACH , 2010 .

[111]  Ali Karami,et al.  Persian sign language (PSL) recognition using wavelet transform and neural networks , 2011, Expert Syst. Appl..

[112]  Jakub Segen,et al.  Gesture VR: vision-based 3D hand interace for spatial interaction , 1998, MULTIMEDIA '98.

[113]  J. P. Lewis,et al.  SmartCanvas: a gesture-driven intelligent drawing desk system , 2005, IUI.

[114]  Ronald Poppe,et al.  Vision-based human motion analysis: An overview , 2007, Comput. Vis. Image Underst..

[115]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[116]  Richard L. Zunkel.  Hand Geometry Based Verification , 1996 .

[117]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[118]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[119]  Thomas S. Huang,et al.  Static Hand Gesture Recognition based on Local Orientation Histogram Feature Distribution Model , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[120]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[121]  Tamás Szirányi,et al.  Dynamic training of hand gesture recognition system , 2004, ICPR 2004.

[122]  Yoshihiko Nakamura,et al.  Motion recognition and recovery from occluded monocular observations , 2014, Robotics Auton. Syst..

[123]  James L. Crowley,et al.  Active hand tracking , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[124]  Mauro Donadeo,et al.  Combining multiple depth-based descriptors for hand gesture recognition , 2014, Pattern Recognit. Lett..

[125]  Sumantra Dutta Roy,et al.  Hand gesture modelling and recognition involving changing shapes and trajectories, using a Predictive EigenTracker , 2007, Pattern Recognit. Lett..

[126]  Wei Zhang,et al.  A Scale and Rotation Invariant Interest Points Detector Based on Gabor Filters , 2009, FGIT-SIP.

[127]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.

[128]  Dmitry B. Goldgof,et al.  Understanding Transit Scenes: A Survey on Human Behavior-Recognition Algorithms , 2010, IEEE Transactions on Intelligent Transportation Systems.

[129]  Yousra Ben Jemaa,et al.  Automatic local Gabor Features extraction for face recognition , 2009, ArXiv.

[130]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[131]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[132]  Ai Poh Loh,et al.  Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds , 2012, International Journal of Computer Vision.

[133]  Francis K. H. Quek.  Unencumbered Gestural Interaction , 1996, IEEE Multim..

[134]  Qing Chen,et al.  Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar , 2008, IEEE Transactions on Instrumentation and Measurement.

[135]  Hossam Ali,et al.  Hand Gesture Recognition Using Modified 1$ and Background Subtraction Algorithms , 2015 .

[136]  Yoichi Sato,et al.  Integrating paper and digital information on EnhancedDesk: a method for realtime finger tracking on an augmented desk system , 2001, TCHI.

[137]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[138]  Chengjun Liu,et al.  Gabor-based kernel PCA with fractional power polynomial models for face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[139]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[140]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, CAIP.

[141]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[142]  Shuzhi Sam Ge,et al.  Hand Gesture Recognition and Tracking based on Distributed Locally Linear Embedding , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.