Averaging of motion capture recordings for movements’ templates generation

In this paper we propose, describe and evaluate the novel motion capture (MoCap) data averaging framework. It incorporates hierarchical kinematic model, angle coordinates’ preprocessing methods, that recalculate the original MoCap recording making it applicable for further averaging algorithms, and finally signals averaging processing. We have tested two signal averaging methods namely Kalman Filter (KF) and Dynamic Time Warping barycenter averaging (DBA). The propose methods have been tested on MoCap recordings of elite Karate athlete, multiple champion of Oyama karate knockdown kumite who performed 28 different karate techniques repeated 10 times each. The proposed methods proved to have not only high effectiveness measured with root-mean-square deviation (4.04 ± 5.03 degrees for KF and 5.57 ± 6.27 for DBA) and normalized Dynamic Time Warping distance (0.90 ± 1.58 degrees for KF and 0.93 ± 1.23 for DBA), but also the reconstruction and visualization of those recordings persists all crucial aspects of those complicated actions. The proposed methodology has many important applications in classification, clustering, kinematic analysis and coaching. Our approach generates an averaged full body motion template that can be practically used for example for human actions recognition. In order to prove it we have evaluated templates generated by our method in human action classification tasks using DTW classifier. We have made two experiments. In first leave - one - out cross - validation we have obtained 100% correct recognitions. In second experiment when we classified recordings of one person using templates of another recognition rate 94.2% was obtained.

[1]  Steffen Masik,et al.  Usage of Virtual Reality Technology to Study Reactions in Karate-Kumite , 2012 .

[2]  Pierre Gançarski,et al.  A global averaging method for dynamic time warping, with applications to clustering , 2011, Pattern Recognit..

[3]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[4]  Wenyu Zhang,et al.  Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[5]  Adam Piórkowski,et al.  Creating 3D Web-Based Viewing Services for DICOM Images , 2009, CN.

[6]  Jenq-Neng Hwang,et al.  A Review on Video-Based Human Activity Recognition , 2013, Comput..

[7]  Johnny Padulo,et al.  Effect of different knee starting angles on intersegmental coordination and performance in vertical jumps. , 2015, Human movement science.

[8]  Jean-Pierre Devogelaer,et al.  Balancing benefits and risks of glucocorticoids in rheumatic diseases and other inflammatory joint disorders: new insights from emerging data. An expert consensus paper from the European Society for Clinical and Economic Aspects of Osteoporosis and Osteoarthritis (ESCEO) , 2016, Aging Clinical and Experimental Research.

[9]  Marzena Bielecka,et al.  Automatized fuzzy evaluation of CT scan heart slices for creating 3D/4D heart model , 2015, Appl. Soft Comput..

[10]  Bülent Sankur,et al.  Graph-based analysis of physical exercise actions , 2013, MIIRH '13.

[11]  Joan Lasenby,et al.  Estimating missing marker positions using low dimensional Kalman smoothing. , 2016, Journal of biomechanics.

[12]  Valentina Camomilla,et al.  Neuromuscular control adaptations in elite athletes: the case of top level karateka , 2010, European Journal of Applied Physiology.

[13]  Sebastian Nowozin,et al.  Efficient Nonlinear Markov Models for Human Motion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[15]  Hazem Wannous,et al.  3D Human Video Retrieval: from Pose to Motion Matching , 2013, 3DOR@Eurographics.

[16]  Marcio Magini,et al.  Electromiographic and kinematic characteristics of Kung Fu Yau-Man palm strike. , 2008, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[17]  Mario Fernando Montenegro Campos,et al.  Online gesture recognition from pose kernel learning and decision forests , 2014, Pattern Recognit. Lett..

[18]  Petros Maragos,et al.  Human action recognition using Histographic methods and hidden Markov models for visual martial arts applications , 2012, 2012 19th IEEE International Conference on Image Processing.

[19]  Xiaodong Yang,et al.  Effective 3D action recognition using EigenJoints , 2014, J. Vis. Commun. Image Represent..

[20]  Francesco Tisato,et al.  Karate moves recognition from skeletal motion , 2013, Electronic Imaging.

[21]  Meltem Izzetoglu,et al.  Motion artifact cancellation in NIR spectroscopy using discrete Kalman filtering , 2010, Biomedical engineering online.

[22]  Jesús Francisco Vargas-Bonilla,et al.  HMM and DTW for evaluation of therapeutical gestures using kinect , 2016, ArXiv.

[23]  Cesare Furlanello,et al.  Combining feature selection and DTW for time-varying functional genomics , 2006, IEEE Transactions on Signal Processing.

[24]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Markus H. Gross,et al.  Combining body sensors and visual sensors for motion training , 2005, ACE '05.

[26]  Marcus Fraga Vieira,et al.  Effects of general fatigue induced by incremental maximal exercise test on gait stability and variability of healthy young subjects. , 2016, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[27]  Guodong Liu,et al.  Estimation of missing markers in human motion capture , 2006, The Visual Computer.

[28]  Marek R. Ogiela,et al.  Human actions recognition on multimedia hardware using angle-based and coordinate-based features and multivariate continuous hidden Markov model classifier , 2016, Multimedia Tools and Applications.

[29]  Chuan-Jun Su,et al.  Kinect-enabled home-based rehabilitation system using Dynamic Time Warping and fuzzy logic , 2014, Appl. Soft Comput..

[30]  Vincent G Kelly,et al.  Towards a Determination of the Physiological Characteristics Distinguishing Successful Mixed Martial Arts Athletes: A Systematic Review of Combat Sport Literature , 2016, Sports Medicine.

[31]  Irene Cheng,et al.  Perceptually Guided Fast Compression of 3-D Motion Capture Data , 2011, IEEE Transactions on Multimedia.

[32]  Tarik Arici,et al.  Robust gesture recognition using feature pre-processing and weighted dynamic time warping , 2014, Multimedia Tools and Applications.

[33]  Adam Glowacz,et al.  Recognition of images of finger skin with application of histogram, image filtration and K-NN classifier , 2016 .

[34]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Zheng H. Zhu,et al.  Autonomous robotic capture of non-cooperative target using visual servoing and motion predictive control , 2014, Auton. Robots.

[36]  Abdul Halim Mokhtar,et al.  Gait analysis of national athletes after anterior cruciate ligament reconstruction following three stages of rehabilitation program: Symmetrical perspective. , 2016, Gait & posture.

[37]  Marek R. Ogiela,et al.  Semantic Description and Recognition of Human Body Poses and Movement Sequences with Gesture Description Language , 2012, FGIT-MulGraB/BSBT/IUrC.

[38]  Richard Kulpa,et al.  Which technology to investigate visual perception in sport: video vs. virtual reality. , 2015, Human movement science.

[39]  Hiroshi Hashimoto,et al.  The curved surface visualization of the expert behavior for skill transfer using Microsoft Kinect , 2014, 2014 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[40]  Valentina Camomilla,et al.  Differences in neuromuscular control between impact and no impact roundhouse kick in athletes of different skill levels. , 2013, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[41]  Mauro Gonçalves,et al.  Neuromuscular performance of Bandal Chagui: Comparison of subelite and elite taekwondo athletes. , 2016, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[42]  Shuang Wang,et al.  A Review on Human Activity Recognition Using Vision-Based Method , 2017, Journal of healthcare engineering.

[43]  Osmar Pinto Neto,et al.  The effect of hand dominance on martial arts strikes. , 2012, Human movement science.

[44]  Ling Chen,et al.  Hierarchical Complex Activity Representation and Recognition Using Topic Model and Classifier Level Fusion. , 2017, IEEE transactions on bio-medical engineering.

[45]  ChenXi,et al.  Skeleton-based action recognition with extreme learning machines , 2015 .

[46]  Markus Koskela,et al.  Skeleton-based action recognition with extreme learning machines , 2015, Neurocomputing.

[47]  Stefano Berretti,et al.  Representation, Analysis, and Recognition of 3D Humans , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[48]  Marek R. Ogiela,et al.  Human Actions Modelling and Recognition in Low-Dimensional Feature Space , 2015, 2015 10th International Conference on Broadband and Wireless Computing, Communication and Applications (BWCCA).

[49]  Jin Young Choi,et al.  Hierarchical Kalman-particle filter with adaptation to motion changes for object tracking , 2011, Comput. Vis. Image Underst..

[50]  Laurent Étienne,et al.  Trajectory Box Plot; a New Pattern to Summarize Movements , 2015 .

[51]  Wolfram Burgard,et al.  Graph-Based Action Models for Human Motion Classification , 2012, ROBOTIK.

[52]  Richard Kulpa,et al.  Using virtual humans and computer animations to learn complex motor skills: a case study in karate , 2011 .

[53]  Maja J. Mataric,et al.  Automated Proxemic Feature Extraction and Behavior Recognition: Applications in Human-Robot Interaction , 2013, Int. J. Soc. Robotics.

[54]  Marek R. Ogiela,et al.  Application of Assistive Computer Vision Methods to Oyama Karate Techniques Recognition , 2015, Symmetry.

[55]  Ju Jin,et al.  The adaptive Kalman filter based on fuzzy logic for inertial motion capture system , 2014 .

[56]  Adam Glowacz,et al.  Recognition of Acoustic Signals of Synchronous Motors with the Use of MoFS and Selected Classifiers , 2015 .

[57]  Thomas B. Schön,et al.  An optimization-based approach to human body motion capture using inertial sensors , 2014 .

[58]  Christian Ritz,et al.  Motion classification using Dynamic Time Warping , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[59]  Cheong Boon Soh,et al.  A wearable wireless ultrasonic sensor network for human arm motion tracking , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[60]  Wan Fatimah Wan Ahmad,et al.  Static Hand Gesture Recognition Using Local Gabor Filter , 2012 .

[61]  Marc Christie,et al.  Thinking Like a Director , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[62]  李英民,et al.  Instantaneous spectrum estimation of earthquake ground motions based on unscented Kalman filter method , 2007 .

[63]  Marek R. Ogiela,et al.  Full body movements recognition - unsupervised learning approach with heuristic R-GDL method , 2015, Digit. Signal Process..

[64]  Veronika Weinbeer,et al.  Changes in dual-task performance after 5 months of karate and fitness training for older adults to enhance fall prevention , 2016, Aging Clinical and Experimental Research.

[65]  Orlando Fernandes,et al.  Kinematic and electromyographic analyses of a karate punch. , 2011, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[66]  J. Helske,et al.  KFAS: Exponential Family State Space Models in R , 2016, 1612.01907.

[67]  Adolfo López,et al.  Model-based recognition of human actions by trajectory matching in phase spaces , 2012, Image Vis. Comput..

[68]  Pham Chinh Huu,et al.  Human Action Recognition Using Dynamic Time Warping and Voting Algorithm (1) , 2014 .

[69]  Toni Giorgino,et al.  Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package , 2009 .

[70]  Soon Ki Jung,et al.  Synthesis of Human Motion Using Kalman Filter , 1998, CAPTECH.