Application of Assistive Computer Vision Methods to Oyama Karate Techniques Recognition

In this paper we propose a novel algorithm that enables online actions segmentation and classification. The algorithm enables segmentation from an incoming motion capture (MoCap) data stream, sport (or karate) movement sequences that are later processed by classification algorithm. The segmentation is based on Gesture Description Language classifier that is trained with an unsupervised learning algorithm. The classification is performed by continuous density forward-only hidden Markov models (HMM) classifier. Our methodology was evaluated on a unique dataset consisting of MoCap recordings of six Oyama karate martial artists including multiple champion of Kumite Knockdown Oyama karate. The dataset consists of 10 classes of actions and included dynamic actions of stands, kicks and blocking techniques. Total number of samples was 1236. We have examined several HMM classifiers with various number of hidden states and also Gaussian mixture model (GMM) classifier to empirically find the best setup of the proposed method in our dataset. We have used leave-one-out cross validation. The recognition rate of our methodology differs between karate techniques and is in the range of 81% ± 15% even to 100%. Our method is not limited for this class of actions but can be easily adapted to any other MoCap-based actions. The description of our approach and its evaluation are the main contributions of this paper. The results presented in this paper are effects of pioneering research on online karate action classification.

[1]  Marek R. Ogiela,et al.  Rule-based approach to recognizing human body poses and gestures in real time , 2013, Multimedia Systems.

[2]  Francesco Tisato,et al.  Karate moves recognition from skeletal motion , 2013, Electronic Imaging.

[3]  Richard Kulpa,et al.  Which technology to investigate visual perception in sport: video vs. virtual reality. , 2015, Human movement science.

[4]  Maja J. Mataric,et al.  Automated Proxemic Feature Extraction and Behavior Recognition: Applications in Human-Robot Interaction , 2013, Int. J. Soc. Robotics.

[5]  Bülent Sankur,et al.  Graph-based analysis of physical exercise actions , 2013, MIIRH '13.

[6]  Piotr Jagodziński,et al.  Assessment of Application Technology of Natural User Interfaces in the Creation of a Virtual Chemical Laboratory , 2015 .

[7]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[8]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[9]  Ehud Rivlin,et al.  Online action recognition using covariance of shape and motion , 2014, Comput. Vis. Image Underst..

[10]  Jørgen Kongsro,et al.  Estimation of pig weight using a Microsoft Kinect prototype imaging system , 2014 .

[11]  Martin A. Giese,et al.  Estimation of Skill Levels in Sports Based on Hierarchical Spatio-Temporal Correspondences , 2003, DAGM-Symposium.

[12]  Adam Glowacz,et al.  Diagnostics of Direct Current motor with application of acoustic signals, reflection coefficients and K-Nearest Neighbor classifier , 2012 .

[13]  Linda Denehy,et al.  Validity of the Microsoft Kinect for assessment of postural control. , 2012, Gait & posture.

[14]  Yonggwan Won,et al.  3D motion matching algorithm using signature feature descriptor , 2014, Multimedia Tools and Applications.

[15]  Marek R. Ogiela,et al.  Dependence of Kinect sensors number and position on gestures recognition with Gesture Description Language semantic classifier , 2013, 2013 Federated Conference on Computer Science and Information Systems.

[16]  P. Olivier,et al.  Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. , 2014, Gait & posture.

[17]  Qunsheng Peng,et al.  Online robust action recognition based on a hierarchical model , 2014, The Visual Computer.

[18]  Einoshin Suzuki,et al.  Classifying actions based on histogram of oriented velocity vectors , 2014, Journal of Intelligent Information Systems.

[19]  Dimitris Kastaniotis,et al.  Pose-based human action recognition via sparse representation in dissimilarity space , 2014, J. Vis. Commun. Image Represent..

[20]  Xinhao Xu,et al.  From psychomotor to ‘motorpsycho’: learning through gestures with body sensory technologies , 2014, Educational Technology Research and Development.

[21]  Chuan-Jun Su,et al.  Kinect-enabled home-based rehabilitation system using Dynamic Time Warping and fuzzy logic , 2014, Appl. Soft Comput..

[22]  Tarik Arici,et al.  Robust gesture recognition using feature pre-processing and weighted dynamic time warping , 2014, Multimedia Tools and Applications.

[23]  Sotirios Chatzis,et al.  A conditional random field-based model for joint sequence segmentation and classification , 2013, Pattern Recognit..

[24]  Adolfo López,et al.  Model-based recognition of human actions by trajectory matching in phase spaces , 2012, Image Vis. Comput..

[25]  Stepán Obdrzálek,et al.  Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[26]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[28]  Marek R. Ogiela,et al.  Natural User Interfaces in Medical Image Analysis , 2015, Advances in Computer Vision and Pattern Recognition.

[29]  Mathieu Barnachon,et al.  Ongoing human action recognition with motion capture , 2014, Pattern Recognit..

[30]  Jeremy N. Bailenson,et al.  Automatically Detected Nonverbal Behavior Predicts Creativity in Collaborating Dyads , 2014 .

[31]  Adam Glowacz,et al.  Dc machine diagnostics based on sound recognition with application of FFT and fuzzy logic , 2008 .

[32]  Fernando De la Torre,et al.  Joint segmentation and classification of human actions in video , 2011, CVPR 2011.

[33]  Zhijun Li,et al.  Development of human-machine interface for teleoperation of a mobile manipulator , 2012 .

[34]  Marek R. Ogiela,et al.  Full-body gestures and movements recognition: user descriptive and unsupervised learning approaches in GDL classifier , 2014, Optics & Photonics - Optical Engineering + Applications.

[35]  Fan Zhong,et al.  Estimation of Kinect depth confidence through self-training , 2014, The Visual Computer.

[36]  Xiaodong Yang,et al.  Effective 3D action recognition using EigenJoints , 2014, J. Vis. Commun. Image Represent..

[37]  Adrien Chan-Hon-Tong,et al.  Simultaneous segmentation and classification of human actions in video streams using deeply optimized Hough transform , 2014, Pattern Recognit..

[38]  Dimitrios Makris,et al.  Fall detection system using Kinect’s infrared sensor , 2014, Journal of Real-Time Image Processing.

[39]  Mario Fernando Montenegro Campos,et al.  Online gesture recognition from pose kernel learning and decision forests , 2014, Pattern Recognit. Lett..

[40]  M. Alizadeh,et al.  Design of a marker-based human motion tracking system , 2007, Biomed. Signal Process. Control..

[41]  Martin Kampel,et al.  Introducing the use of depth data for fall detection , 2013, Personal and Ubiquitous Computing.

[42]  Clementine Nyirarugira,et al.  Stratified gesture recognition using the normalized longest common subsequence with rough sets , 2015, Signal Process. Image Commun..

[43]  Tomasz Hachaj,et al.  Knowledge Bricks - Educational immersive reality environment , 2015, Int. J. Inf. Manag..

[44]  Petros Maragos,et al.  Human action recognition using Histographic methods and hidden Markov models for visual martial arts applications , 2012, 2012 19th IEEE International Conference on Image Processing.

[45]  Quoc V. Le,et al.  Tiled convolutional neural networks , 2010, NIPS.