Stratified gesture recognition using the normalized longest common subsequence with rough sets

In this paper, we propose a stratified gesture recognition method that integrates rough set theory with the longest common subsequence method to classify free-air gestures, for natural human-computer interaction. Gesture vocabularies are often composed of gestures that are highly correlated or comprise gestures that are a proper part of others. This reduces the accuracy of most classifiers if no further actions are taken. In this paper, gestures are encoded in orientation segments which facilitate their analysis and reduce the processing time. To improve the accuracy of gesture recognition on ambiguous gestures, we generate rough set decision tables conditioned on the longest common subsequences; the decision tables store discriminative information on ambiguous gestures. We efficiently perform stratified gesture recognition in two steps: first a gesture is classified in its equivalence class, under a predefined rough set indiscernibility, and then it is recognized using the normalized longest common subsequence paired with rough set decision tables. Experimental results show an improvement of the recognition rate of the longest common subsequence; on preisolated gestures, we achieve an improvement of 6.06% and 15.09%, and on stream gestures 19.79% and 28.4% on digit and alphabet gesture vocabularies, respectively. HighlightsAn integration of rough set theory to the longest common subsequence to classify dynamic hand gestures is discussed.Gesture vocabulary analysis using rough sets reveals the decision attributes that will correctly classify ambiguous gestures.A segment matching that reduces considerably the size of LCS matrices to be computed is presented.

[1]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Alfred V. Aho,et al.  Bounds on the Complexity of the Longest Common Subsequence Problem , 1976, J. ACM.

[3]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Helman Stern,et al.  Most Probable Longest Common Subsequence for Recognition of Gesture Character Input , 2013, IEEE Transactions on Cybernetics.

[5]  Helman Stern,et al.  Most discriminating segment - Longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification , 2013, Pattern Recognit. Lett..

[6]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory , 2008, 2008 19th International Conference on Pattern Recognition.

[7]  Vassilis Athitsos,et al.  Comparing gesture recognition accuracy using color and depth information , 2011, PETRA '11.

[8]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[9]  Cheng-Lin Liu,et al.  Visual Gesture Character String Recognition by Classification-Based Segmentation with Stroke Deletion , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[10]  Hyo-Rim Choi,et al.  Modified levenshtein distance for real-time gesture recognition , 2013, 2013 6th International Congress on Image and Signal Processing (CISP).

[11]  Qiang Shen,et al.  Exploring the boundary region of tolerance rough sets for feature selection , 2009, Pattern Recognit..

[12]  Dimitrios Gunopulos,et al.  Discovering similar multidimensional trajectories , 2002, Proceedings 18th International Conference on Data Engineering.

[13]  Romero Tori,et al.  Design and Implementation of a Flexible Hand Gesture Command Interface for Games Based on Computer Vision , 2009, 2009 VIII Brazilian Symposium on Games and Digital Entertainment.

[14]  Mohan M. Trivedi,et al.  Learning trajectory patterns by clustering: Experimental studies and comparative evaluation , 2009, CVPR.

[15]  Zbigniew Suraj,et al.  An Introduction to Rough Set Theory and Its Applications A tutorial , 2004 .

[16]  Daniel Kelly,et al.  Evaluation of threshold model HMMS and Conditional Random Fields for recognition of spatiotemporal gestures in sign language , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[17]  Sergio Escalera,et al.  Probability-Based Dynamic Time Warping for Gesture Recognition on RGB-D Data , 2012, WDIA.

[18]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Stan Sclaroff,et al.  Accurate and Efficient Gesture Spotting via Pruning and Subgesture Reasoning , 2005, ICCV-HCI.

[20]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[21]  Ayoub Al-Hamadi,et al.  Robust methods for hand gesture spotting and recognition using Hidden Markov Models and Conditional Random Fields , 2010, The 10th IEEE International Symposium on Signal Processing and Information Technology.

[22]  Yunhui Liu,et al.  2D Human Gesture Tracking and Recognition by the Fusion of MEMS Inertial and Vision Sensors , 2014, IEEE Sensors Journal.

[23]  Wei Hu,et al.  Automatic user state recognition for hand gesture based low-cost television control system , 2014, IEEE Transactions on Consumer Electronics.

[24]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.