Combined Dynamic Time Warping with Multiple Sensors for 3D Gesture Recognition

Cyber-physical systems, which closely integrate physical systems and humans, can be applied to a wider range of applications through user movement analysis. In three-dimensional (3D) gesture recognition, multiple sensors are required to recognize various natural gestures. Several studies have been undertaken in the field of gesture recognition; however, gesture recognition was conducted based on data captured from various independent sensors, which rendered the capture and combination of real-time data complicated. In this study, a 3D gesture recognition method using combined information obtained from multiple sensors is proposed. The proposed method can robustly perform gesture recognition regardless of a user’s location and movement directions by providing viewpoint-weighted values and/or motion-weighted values. In the proposed method, the viewpoint-weighted dynamic time warping with multiple sensors has enhanced performance by preventing joint measurement errors and noise due to sensor measurement tolerance, which has resulted in the enhancement of recognition performance by comparing multiple joint sequences effectively.

[1]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[2]  Dimitrios Makris,et al.  G3D: A gaming action dataset and real time action recognition evaluation framework , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[4]  D. Prashanthb,et al.  Robust gesture recognition using Kinect: A comparison between DTW and HMM , 2015 .

[5]  Ankit Chaudhary,et al.  Robust gesture recognition using Kinect: A comparison between DTW and HMM , 2015 .

[6]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[7]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[8]  I.A. Essa,et al.  Ubiquitous sensing for smart and aware environments , 2000, IEEE Wirel. Commun..

[9]  Michael Barlow,et al.  An Evaluation of DTW Approaches for Whole-of-Body Gesture Recognition , 2014, BCS HCI.

[10]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Christos Faloutsos,et al.  FTW: fast similarity search under the time warping distance , 2005, PODS.

[12]  Jonathan W. Decker,et al.  Performance measurements for the Microsoft Kinect skeleton , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[13]  Daniel Lemire,et al.  Faster retrieval with a two-pass dynamic-time-warping lower bound , 2008, Pattern Recognit..

[14]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[15]  Tim Roberts,et al.  Multi-Kinect Tracking for Dismounted Soldier Training , 2012 .

[16]  P. Olivier,et al.  Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. , 2014, Gait & posture.

[17]  Greg Borenstein,et al.  Making Things See: 3D vision with Kinect, Processing, Arduino, and MakerBot , 2012 .

[18]  Joseph J. LaViola,et al.  Exploring the Benefits of Context in 3D Gesture Recognition for Game-Based Virtual Environments , 2015, ACM Trans. Interact. Intell. Syst..

[19]  Wenbing Zhao,et al.  A Survey of Applications and Human Motion Recognition with Microsoft Kinect , 2015, Int. J. Pattern Recognit. Artif. Intell..

[20]  Jae-Hean Kim,et al.  Calibration of multi-Kinect and multi-camera setup for full 3D reconstruction , 2013, IEEE ISR 2013.

[21]  Eamonn Keogh.  Exact Indexing of Dynamic Time Warping , 2002, VLDB.