Gesture Recognition Performance Score: A New Metric to Evaluate Gesture Recognition Systems

In spite of many choices available for gesture recognition algorithms, the selection of a proper algorithm for a specific application remains a difficult task. The available algorithms have different strengths and weaknesses making the matching between algorithms and applications complex. Accurate evaluation of the performance of a gesture recognition algorithm is a cumbersome task. Performance evaluation by recognition accuracy alone is not sufficient to predict its successful real-world implementation. We developed a novel Gesture Recognition Performance Score (\(GRPS\)) for ranking gesture recognition algorithms, and to predict the success of these algorithms in real-world scenarios. The \(GRPS\) is calculated by considering different attributes of the algorithm, the evaluation methodology adopted, and the quality of dataset used for testing. The \(GRPS\) calculation is illustrated and applied on a set of vision based hand/ arm gesture recognition algorithms reported in the last 15 years. Based on \(GRPS\) a ranking of hand gesture recognition algorithms is provided. The paper also presents an evaluation metric namely Gesture Dataset Score (\(GDS\)) to quantify the quality of gesture databases. The \(GRPS\) calculator and results are made publicly available (http://software.ihpc.a-star.edu.sg/grps/).

[1]  Janusz Konrad,et al.  A gesture-driven computer interface using Kinect , 2012, 2012 IEEE Southwest Symposium on Image Analysis and Interpretation.

[2]  Aditya Ramamoorthy,et al.  Recognition of dynamic hand gestures , 2003, Pattern Recognit..

[3]  Sébastien Marcel,et al.  Hand posture recognition in a body-face centered space , 1999, CHI Extended Abstracts.

[4]  Xindong Wu,et al.  RIEVL: Recursive Induction Learning in Hand Gesture Recognition , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[6]  Sergio Escalera,et al.  Multi-modal gesture recognition challenge 2013: dataset and results , 2013, ICMI '13.

[7]  Ling Shao,et al.  Learning Discriminative Representations from RGB-D Video Data , 2013, IJCAI.

[8]  Bian Wu,et al.  A hand gesture recognition system based on local linear embedding , 2005, J. Vis. Lang. Comput..

[9]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Surendra Ranganath,et al.  Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Bharti Bansal,et al.  Gesture Recognition: A Survey , 2016 .

[12]  Biing-Hwang Juang,et al.  6DMG: a new 6D motion gesture database , 2012, MMSys '12.

[13]  Ai Poh Loh,et al.  Hand Posture and Face Recognition Using a Fuzzy-Rough Approach , 2010, Int. J. Humanoid Robotics.

[14]  Neil A. Thacker,et al.  Performance characterization in computer vision: A guide to best practices , 2008, Comput. Vis. Image Underst..

[15]  Luc Van Gool,et al.  Real-time 3D hand gesture interaction with a robot for understanding directions from humans , 2011, 2011 RO-MAN.

[16]  Elena Mugellini,et al.  ChAirGest: a challenge for multimodal mid-air gesture recognition for close HCI , 2013, ICMI '13.

[17]  Agnès Just,et al.  HMM and IOHMM for the Recognition of Mono- and Bi-Manual 3D Hand Gestures , 2004, BMVC.

[18]  Jochen Triesch,et al.  Robust classification of hand postures against complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[19]  Paul Lukowicz,et al.  Performance metrics for activity recognition , 2011, TIST.

[20]  Jochen Triesch,et al.  A gesture interface for human-robot-interaction , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[21]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[22]  Jochen Triesch,et al.  A System for Person-Independent Hand Posture Recognition against Complex Backgrounds , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[24]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[25]  Geoffrey E. Hinton,et al.  Learning Generative Texture Models with extended Fields-of-Experts , 2009, BMVC.

[26]  Venu Govindaraju,et al.  A temporal Bayesian model for classifying, detecting and localizing activities in video sequences , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[27]  Ling Shao,et al.  One shot learning gesture recognition from RGBD images , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[28]  Shuzhi Sam Ge,et al.  Hand Gesture Recognition and Tracking based on Distributed Locally Linear Embedding , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[29]  Tamás Szirányi,et al.  User-adaptive hand gesture recognition system with interactive training , 2005, Image Vis. Comput..

[30]  Agnès Just,et al.  A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition , 2009, Comput. Vis. Image Underst..

[31]  Larry S. Davis,et al.  Recognizing actions by shape-motion prototype trees , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  Gang Hua,et al.  Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields , 2012, Image Vis. Comput..

[33]  Yui Man Lui,et al.  A least squares regression framework on manifolds and its application to gesture recognition , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[34]  Helman Stern,et al.  Most Probable Longest Common Subsequence for Recognition of Gesture Character Input , 2013, IEEE Transactions on Cybernetics.

[35]  Isabelle Guyon,et al.  ChaLearn gesture challenge: Design and first results , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[36]  Tae-Kyun Kim,et al.  Tensor Canonical Correlation Analysis for Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Paolo Dario,et al.  A Survey of Glove-Based Systems and Their Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[38]  Ai Poh Loh,et al.  Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds , 2012, International Journal of Computer Vision.

[39]  Sumantra Dutta Roy,et al.  Hand gesture modelling and recognition involving changing shapes and trajectories, using a Predictive EigenTracker , 2007, Pattern Recognit. Lett..

[40]  Yale Song,et al.  Tracking body and hands for gesture recognition: NATOPS aircraft handling signals database , 2011, Face and Gesture 2011.

[41]  Dmitry B. Goldgof,et al.  Gesture recognition using Bezier curves for visualization navigation from registered 3-D data , 2004, Pattern Recognit..

[42]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Hafiz Imtiaz,et al.  A template matching approach of one-shot-learning gesture recognition , 2013, Pattern Recognit. Lett..

[44]  Helena M. Mentis,et al.  Instructing people for training gestural interactive systems , 2012, CHI.

[45]  Lale Akarun,et al.  Randomized decision forests for static and dynamic hand shape classification , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[46]  Wu-Chih Hu,et al.  Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination , 2011, Expert Syst. Appl..