Human Gesture Recognition using Keyframes on Local Joint Motion Trajectories

Human Action Recognition (HAR) systems are systems that recognize and classify the actions that users perform against the sensor or camera. In most HAR systems, an input test data is compared with the reference data in the database using various methods. Classification process is performed according to the result obtained. The size of the test or reference data directly affects the operation speed of the system. Reduced data size allows a significant performance increase in system operation speed. In this study, action recognition method is proposed by using skeletal joint information obtained by Microsoft Kinect sensor. Splitting keyframes are obtained from the skeletal joint information. The keyframes are observed as a distinguishing feature. Therefore, these keyframes are used for the classification process. Keeping the keyframes instead of keeping the position or angle information of action in the reference database can benefit from memory and working time. The weight value of each keyframes is calculated in the method. The problem of temporal differences that occur when comparing test and reference action is solved by Dynamic Time Warping (DTW). The k-nearest neighbor’s algorithm is used for classification according to the obtained results from DTW. The sample has been tested in a data set so that the success of the method can be tested. As a result, 100% correct classification was achieved. It is also suitable for working at real time systems. Breakpoints can also be used to provide feedback to the user as a result of the classification process. The magnitude and direction of the keyframes, the change in the trajectory of joint, the position and the time of its existence also give information about the time errors.

[1]  Xinyu Wu,et al.  Dimensionality reduction of data sequences for human activity recognition , 2016, Neurocomputing.

[2]  Junsong Yuan,et al.  Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction , 2011, 2011 8th International Conference on Information, Communications & Signal Processing.

[3]  Joseph J. LaViola,et al.  Exploring the Trade-off Between Accuracy and Observational Latency in Action Recognition , 2013, International Journal of Computer Vision.

[4]  Ana M. Barbancho,et al.  Fast-gesture recognition and classification using Kinect: an application for a virtual reality drumkit , 2015, Multimedia Tools and Applications.

[5]  Tarik Arici,et al.  Robust gesture recognition using feature pre-processing and weighted dynamic time warping , 2014, Multimedia Tools and Applications.

[6]  Marcelo R. Campo,et al.  Easy gesture recognition for Kinect , 2014, Adv. Eng. Softw..

[7]  Andrés Pérez-Uribe,et al.  Indoor Activity Recognition by Combining One-vs.-All Neural Network Classifiers Exploiting Wearable and Depth Sensors , 2013, IWANN.

[8]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Binlong Li,et al.  Activity recognition using dynamic subspace angles , 2011, CVPR 2011.

[11]  Xiaodong Yang,et al.  EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[13]  Stephen J. Maybank,et al.  Action classification using a discriminative multilevel HDP-HMM , 2015, Neurocomputing.

[14]  David Sinclair,et al.  A Self-Referential Perceptual Inference Framework for Video Interpretation , 2003, ICVS.

[15]  Cyrus Shahabi,et al.  A PCA-based similarity measure for multivariate time series , 2004, MMDB '04.

[16]  Mohammed Imamul Hassan Bhuiyan,et al.  Classification of motor imagery movements using multivariate empirical mode decomposition and short time Fourier transform based hybrid method , 2016 .

[17]  Tarik Arici,et al.  Gesture Recognition using Skeleton Data with Weighted Dynamic Time Warping , 2013, VISAPP.

[18]  Mario Fernando Montenegro Campos,et al.  Real-Time Gesture Recognition from Depth Data through Key Poses Learning and Decision Forests , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images.

[19]  Martin Masek,et al.  Joint movement similarities for robust 3D action recognition using skeletal data , 2015, J. Vis. Commun. Image Represent..

[20]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[21]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[22]  Darko Kirovski,et al.  Real-time classification of dance gestures from skeleton animation , 2011, SCA '11.

[23]  K. D. Mankoff,et al.  The Kinect: a low‐cost, high‐resolution, short‐range 3D camera , 2013 .