Most discriminating segment - Longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification

In this work, we consider the recognition of dynamic gestures based on representative sub-segments of a gesture, which are denoted as most discriminating segments (MDSs). The automatic extraction and recognition of such small representative segments, rather than extracting and recognizing the full gestures themselves, allows for a more discriminative classifier. A MDS is a sub-segment of a gesture that is most dissimilar to all other gesture sub-segments. Gestures are classified using a MDSLCS algorithm, which recognizes the MDSs using a modified longest common subsequence (LCS) measure. The extraction of MDSs from a data stream uses adaptive window parameters, which are driven by the successive results of multiple calls to the LCS classifier. In a preprocessing stage, gestures that have large motion variations are replaced by several forms of lesser variation. We learn these forms by adaptive clustering of a training set of gestures, where we reemploy the LCS to determine similarity between gesture trajectories. The MDSLCS classifier achieved a gesture recognition rate of 92.6% when tested using a set of pre-cut free hand digit (0-9) gestures, while hidden Markov models (HMMs) achieved an accuracy of 89.5%. When the MDSLCS was tested against a set of streamed digit gestures, an accuracy of 89.6% was obtained. At present the HMMs method is considered the state-of-the-art method for classifying motion trajectories. The MDSLCS algorithm had a higher accuracy rate for pre-cut gestures, and is also more suitable for streamed gestures. MDSLCS provides a significant advantage over HMMs by not requiring data re-sampling during run-time and performing well with small training sets.

[1]  Ching Y. Suen,et al.  New Systems and Architectures for Automatic Speech Recognition and Synthesis , 1987, NATO ASI Series.

[2]  Dimitrios Gunopulos,et al.  Discovering similar multidimensional trajectories , 2002, Proceedings 18th International Conference on Data Engineering.

[3]  Mohan M. Trivedi,et al.  Learning trajectory patterns by clustering: Experimental studies and comparative evaluation , 2009, CVPR.

[4]  Anthony K. H. Tung,et al.  SpADe: On Shape-based Pattern Detection in Streaming Time Series , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[5]  Kazumoto Tanaka.  Gesture Recognition with a Focus on Important Actions by Using a Path Searching Method in Weighted Graph , 2009, ArXiv.

[6]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[7]  C. Y. Chung,et al.  Gesture Recognition Using Hidden Markov Models , 1997 .

[8]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[9]  Helman Stern,et al.  Hand Gesture Recognition for TV Remote Control using Tree-Based Ensemble and LCS Classifiers , 2010, IPCV.

[10]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[11]  Claus Bahlmann,et al.  The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Venu Govindaraju,et al.  A Framework for Hand Gesture Recognition and Spotting Using Sub-gesture Modeling , 2010, 2010 20th International Conference on Pattern Recognition.

[13]  Gian Luca Foresti,et al.  On-line trajectory clustering for anomalous events detection , 2006, Pattern Recognit. Lett..

[14]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[15]  Helman Stern,et al.  Most Probable Longest Common Subsequence for Recognition of Gesture Character Input , 2013, IEEE Transactions on Cybernetics.

[16]  Min Xie,et al.  Vision-Based Hand Gesture Recognition for Human-Vehicle Interaction , 1998 .

[17]  Xueyin Lin,et al.  Toward real-time human-computer interaction with continuous dynamic hand gestures , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[18]  A. Kuzmanic,et al.  Hand shape classification using DTW and LCSS as similarity measures for vision-based gesture recognition system , 2007, EUROCON 2007 - The International Conference on "Computer as a Tool".

[19]  L. Bergroth,et al.  A survey of longest common subsequence algorithms , 2000, Proceedings Seventh International Symposium on String Processing and Information Retrieval. SPIRE 2000.

[20]  Stan Sclaroff,et al.  Accurate and Efficient Gesture Spotting via Pruning and Subgesture Reasoning , 2005, ICCV-HCI.

[21]  Luca Maria Gambardella,et al.  Deep Big Multilayer Perceptrons for Digit Recognition , 2012, Neural Networks: Tricks of the Trade.

[22]  Ruiduo Yang,et al.  Gesture Recognition using Hidden Markov Models from Fragmented Observations , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  M. Trivedi,et al.  Learning trajectory patterns by clustering: Experimental studies and comparative evaluation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jung-Shyr Wu,et al.  Integrating weighted LCS and SVM for 3D handwriting recognition on handheld devices using accelerometers , 2010 .

[25]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Tetsuji Satoh,et al.  Clustering Multidimensional Trajectories based on Shape and Velocity , 2006, 22nd International Conference on Data Engineering Workshops (ICDEW'06).

[27]  Maja Pantic,et al.  Human body gesture recognition using adapted auxiliary particle filtering , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[28]  A. Corradini,et al.  Dynamic time warping for off-line recognition of a small gesture vocabulary , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[29]  Joseph B. Kruskal,et al.  Time Warps, String Edits, and Macromolecules , 1999 .

[30]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[31]  Pritpal Singh,et al.  HANDWRITTEN GURMUKHI CHARACTER RECOGNITION USING WAVELET TRANSFORMS , 2012 .

[32]  J di Martino,et al.  Dynamic time warping algorithms for isolated and connected word recognition , 1987 .

[33]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[34]  Michael J. Fischer,et al.  The String-to-String Correction Problem , 1974, JACM.

[35]  George Kollios,et al.  Extraction and clustering of motion trajectories in video , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[36]  K. Siddharth,et al.  Handwritten Gurmukhi Character Recognition Using Statistical and Background Directional Distribution Features , 2011 .

[37]  Wlodzimierz Kasprzak,et al.  Hand gesture recognition based on free-form contours and probabilistic inference , 2012, Int. J. Appl. Math. Comput. Sci..

[38]  David Sankoff,et al.  Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison , 1983 .

[39]  Alexander K. Seewald,et al.  On the Brittleness of Handwritten Digit Recognition Models , 2012 .

[40]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.