Robust gesture recognition using feature pre-processing and weighted dynamic time warping

Gesture recognition is a technology often used in human-computer interaction applications. Dynamic time warping (DTW) is one of the techniques used in gesture recognition to find an optimal alignment between two sequences. Oftentimes a pre-processing of sequences is required to remove variations due to different camera or body orientations or due to different skeleton sizes between the reference gesture sequences and the test gesture sequences. We discuss a set of pre-processing methods to make the gesture recognition mechanism robust to these variations. DTW computes a dissimilarity measure by time-warping the sequences on a per sample basis by using the distance between the current reference and test sequences. However, all body joints involved in a gesture are not equally important in computing the distance between two sequence samples. We propose a weighted DTW method that weights joints by optimizing a discriminant ratio. Finally, we demonstrate the performance of our pre-processing and the weighted DTW method and compare our results with the conventional DTW and state-of-the-art.

[1]  Sergio Escalera,et al.  Featureweighting in dynamic timewarping for gesture recognition in depth data , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[2]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  A. Corradini,et al.  Dynamic time warping for off-line recognition of a small gesture vocabulary , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[4]  R. Manmatha,et al.  Word image matching using dynamic time warping , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[6]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[7]  Ching Y. Suen,et al.  The State of the Art in Online Handwriting Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  S. Majumder,et al.  Shape, texture and local movement hand gesture features for Indian Sign Language recognition , 2011, 3rd International Conference on Trendz in Information Sciences & Computing (TISC2011).

[9]  Gregory H. Wakefield,et al.  Time Series Alignment for Music Information Retrieval , 2004, ISMIR.

[10]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Tanja Schultz,et al.  HMM-based human motion recognition with optical flow data , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[12]  Trevor Darrell,et al.  Hidden Conditional Random Fields for Gesture Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[14]  Alan Bundy,et al.  Dynamic Time Warping , 1984 .

[15]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[16]  Pavel Senin,et al.  Dynamic Time Warping Algorithm Review , 2008 .

[17]  A. Kuzmanic,et al.  Hand shape classification using DTW and LCSS as similarity measures for vision-based gesture recognition system , 2007, EUROCON 2007 - The International Conference on "Computer as a Tool".

[18]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[19]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[20]  Thomas S. Huang,et al.  Gesture modeling and recognition using finite state machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[21]  Meinard Müller,et al.  Dynamic Time Warping , 2008 .

[22]  Stephen P. Boyd,et al.  Robust Fisher Discriminant Analysis , 2005, NIPS.

[23]  Sukhendu Das,et al.  Real-Time Upper-Body Human Pose Estimation Using a Depth Camera , 2011, MIRAGE.

[24]  Suresh Venkatasubramanian,et al.  Curve Matching, Time Warping, and Light Fields: New Algorithms for Computing Similarity between Curves , 2007, Journal of Mathematical Imaging and Vision.

[25]  R. Shashikant,et al.  Speech Recognition using Dynamic Time Warping , 2013 .

[26]  Tarik Arici,et al.  Gesture Recognition using Skeleton Data with Weighted Dynamic Time Warping , 2013, VISAPP.

[27]  Robert E. Kalaba,et al.  On adaptive control processes , 1959 .

[28]  Carla E. Brodley,et al.  Multivariate decision trees , 2004, Machine Learning.

[29]  Blake Hannaford,et al.  Using Kinect and a Haptic Interface for Implementation of Real-Time Virtual Fixture , 2011 .

[30]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[32]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[33]  Chengdong Wu,et al.  Dynamic hand gesture recognition using motion trajectories and key frames , 2010, 2010 2nd International Conference on Advanced Computer Control.

[34]  L. R. Rabiner,et al.  A comparative study of several dynamic time-warping algorithms for connected-word recognition , 1981, The Bell System Technical Journal.

[35]  D. L. Quam,et al.  Gesture recognition with a DataGlove , 1990, IEEE Conference on Aerospace and Electronics.

[36]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[37]  Andrew D. Wilson.  Using a depth camera as a touch sensor , 2010, ITS '10.

[38]  Michael Hayes,et al.  Altitude control of a quadrotor helicopter using depth map from Microsoft Kinect sensor , 2011, 2011 IEEE International Conference on Mechatronics.

[39]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.

[40]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .