Real-Time 3-D Motion Gesture Recognition using Kinect2 as Basis for Traditional Dance Scripting

This preliminary study presents a system capable of recognizing human gesture in real-time. The gesture is acquired from a Kinect2 sensor which provides skeleton joints represented by three-dimensional coordinate points. The model set consists of eight motion gestures is provided for basis of gesture recognition using Dynamic Time Warping (DTW) algorithm. DTW algorithm is utilized to identify in real time manner by measuring the shortest combined distances in x, y, and z coordinates in order to determined the matched gesture. It can be shown that the system is able to recognize these 8 motions in real time with some limitations. The findings of the this study will provide solid foundation of further research in which the ultimate goal of the research is to create system to automatically recognize sequence of motions in Indonesian traditional dances and convert them into standardized Resource Description Framework (RDF) scripts for the purpose of preserving these dances.

[1]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[2]  Jake K. Aggarwal,et al.  Human detection using depth information by Kinect , 2011, CVPR 2011 WORKSHOPS.

[3]  Z. Liu,et al.  A real time system for dynamic hand gesture recognition with a depth sensor , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[4]  Vassilis Christophides,et al.  Resource Description Framework (RDF) Schema (RDFS) , 2009, Encyclopedia of Database Systems.

[5]  Vassilis Athitsos,et al.  Comparing gesture recognition accuracy using color and depth information , 2011, PETRA '11.

[6]  Yaya Heryadi,et al.  American sign language-based finger-spelling recognition using k-Nearest Neighbors classifier , 2015, 2015 3rd International Conference on Information and Communication Technology (ICoICT).

[7]  Samsu Sempena,et al.  Human action recognition using Dynamic Time Warping , 2011, Proceedings of the 2011 International Conference on Electrical Engineering and Informatics.

[8]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[9]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Nurfitri Anbarsanti,et al.  Dance modelling, learning and recognition system of aceh traditional dance based on hidden Markov model , 2014, 2014 International Conference on Information Technology Systems and Innovation (ICITSI).

[11]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[12]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.

[13]  Song Wang,et al.  Person Identification Using Full-Body Motion and Anthropometric Biometrics from Kinect Videos , 2012, ECCV Workshops.

[14]  Hubert P. H. Shum,et al.  Real-Time Posture Reconstruction for Microsoft Kinect , 2013, IEEE Transactions on Cybernetics.

[15]  A. M. Arymurthy,et al.  A syntactical modeling and classification for performance evaluation of Bali traditional dance , 2012, 2012 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[16]  Noel E. O'Connor,et al.  An advanced virtual dance performance evaluator , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Claudia Linnhoff-Popien,et al.  Gait Recognition with Kinect , 2012 .

[18]  Eamonn J. Keogh,et al.  Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping , 2013, TKDD.

[19]  Stan Salvador,et al.  FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space , 2004 .

[20]  Javid Taheri,et al.  SparseDTW: A Novel Approach to Speed up Dynamic Time Warping , 2009, AusDM.

[21]  Yi Li,et al.  Hand gesture recognition using Kinect , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering.

[22]  Aciek Ida Wuryandari,et al.  Inverse kinematics and gesture pattern recognition using Hidden Markov Model on BeatMe! project: Traditional dance digitalization , 2015, 2015 International Conference on Electrical Engineering and Informatics (ICEEI).

[23]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[24]  Stepán Obdrzálek,et al.  Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  Zeyu Chen,et al.  Gesture Recognition by Using Kinect Skeleton Tracking System , 2013, 2013 5th International Conference on Intelligent Human-Machine Systems and Cybernetics.

[26]  Kanad K. Biswas,et al.  Gesture recognition using Microsoft Kinect® , 2011, The 5th International Conference on Automation, Robotics and Applications.

[27]  L. M. Pedro,et al.  Kinect evaluation for human body movement analysis , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[28]  Arunas Lipnickas,et al.  3D human hand motion recognition system , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[29]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[30]  Tarik Arici,et al.  Gesture Recognition using Skeleton Data with Weighted Dynamic Time Warping , 2013, VISAPP.

[31]  Andreas Widjaja,et al.  Feasibility study of scripting Indonesian traditional dance motion in XML format , 2017, 2017 2nd International conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE).

[32]  Mohamad Ivan Fanany,et al.  Stochastic regular grammar-based learning for basic dance motion recognition , 2013, 2013 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[33]  L. R. Rabiner,et al.  A comparative study of several dynamic time-warping algorithms for connected-word recognition , 1981, The Bell System Technical Journal.

[34]  Ann Hutchinson Guest,et al.  Labanotation : or, Kinetography Laban : the system of analyzing and recording movement , 1970 .

[35]  Petros Daras,et al.  Real-Time Skeleton-Tracking-Based Human Action Recognition Using Kinect Data , 2014, MMM.

[36]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[37]  Meinard Müller,et al.  Dynamic Time Warping , 2008 .