Skeleton-based comparison of throwing motion for handball players

The main goal of this work is to design an automated solution based on RGB-D data for quantitative analysis, perceptible evaluation and comparison of handball player’s performance. To that end, we introduced a new RGB-D dataset that can be used for an objective comparison and evaluation of handball player’s performance during throws. We filmed 62 handball players (44 beginners and 18 experts), who performed the same type of action, using a Kinect V2 sensor that provides RGB data, depth data and skeletons. Moreover, using skeleton data simulating 3D joint connections, we examined the main angles responsible for throwing performance in order to analyze individual skills of handball players (beginners against model and experts) relatively to throw actions. The comparison was performed statically (using only one frame) as well as dynamically during the entire throwing action. In particular, given the temporal sequence of 25 joints of each handball player, we adopted the dynamic time warping technique to compare the throwing motion between two athletes. The obtained results were found to be promising. Thus, the suggested markless solution would help handball coaches to optimize beginners’ movements during throwing actions.

[1]  Hossein Pazhoumand-dar Fuzzy association rule mining for recognising daily activities using Kinect sensors and a single power meter , 2018, J. Ambient Intell. Humaniz. Comput..

[2]  Ross A Clark,et al.  Reliability and concurrent validity of the Microsoft Xbox One Kinect for assessment of standing balance and postural control. , 2015, Gait & posture.

[3]  Jacqueline Alderson A markerless motion capture technique for sport performance analysis and injury prevention: Toward a ‘big data’, machine learning future , 2015 .

[4]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[5]  Roland van den Tillaar,et al.  Is there a proximal-to-distal sequence in overarm throwing in team handball? , 2009, Journal of sports sciences.

[6]  C. V. Jawahar,et al.  Trajectory aligned features for first person action recognition , 2016, Pattern Recognit..

[7]  Joan Lasenby,et al.  Aerodynamic analysis via foreground segmentation , 2017 .

[8]  Sebastian Thrun,et al.  Real-Time Human Pose Tracking from Range Data , 2012, ECCV.

[9]  Pietro Cerveri,et al.  Action Sport Cameras as an Instrument to Perform a 3D Underwater Motion Analysis , 2016, PloS one.

[10]  Yong Du,et al.  Hierarchical recurrent neural network for skeleton based action recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Valentina Camomilla,et al.  Trends Supporting the In-Field Use of Wearable Inertial Sensors for Sport Performance Evaluation: A Systematic Review , 2018, Sensors.

[12]  Balasubramanian Raman,et al.  Evaluating fusion of RGB-D and inertial sensors for multimodal human action recognition , 2020, J. Ambient Intell. Humaniz. Comput..

[13]  Richard Campbell,et al.  Validation of the Microsoft Kinect® camera system for measurement of lower extremity jump landing and squatting kinematics , 2016, Sports biomechanics.

[14]  Marco La Cascia,et al.  3D skeleton-based human action classification: A survey , 2016, Pattern Recognit..

[15]  Naokazu Yokoya,et al.  Summarization of User-Generated Sports Video by Using Deep Action Recognition Features , 2017, IEEE Transactions on Multimedia.

[16]  Mubarak Shah,et al.  Action MACH a spatio-temporal Maximum Average Correlation Height filter for action recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Daijin Kim,et al.  Real-time dance evaluation by markerless human pose estimation , 2018, Multimedia Tools and Applications.

[19]  Matteo Munaro,et al.  Performance evaluation of the 1st and 2nd generation Kinect for multimedia applications , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[20]  Zhou Zhang,et al.  Recent Developments in Game-Based Virtual Reality Educational Laboratories Using the Microsoft Kinect , 2018, Int. J. Emerg. Technol. Learn..

[21]  S. Maxwell,et al.  Bivariate median splits and spurious statistical significance. , 1993 .

[22]  Gonen Eren,et al.  Evaluation of video activity localizations integrating quality and quantity measurements , 2014, Comput. Vis. Image Underst..

[23]  Linda Denehy,et al.  Validity of the Microsoft Kinect for assessment of postural control. , 2012, Gait & posture.

[24]  Wilbert G. Aguilar,et al.  3D Environment Mapping Using the Kinect V2 and Path Planning Based on RRT Algorithms , 2016 .

[25]  Erich Müller,et al.  Skill-dependent proximal-to-distal sequence in team-handball throwing , 2012, Journal of sports sciences.

[26]  J. Cockcroft,et al.  A descriptive study of step alignment and foot positioning relative to the tee by professional rugby union goal-kickers , 2016, Journal of sports sciences.

[27]  Walid Barhoumi Detection of highly articulated moving objects by using co-segmentation with application to athletic video sequences , 2015, Signal Image Video Process..

[28]  Robin Orr,et al.  The validity and intra-tester reliability of markerless motion capture to analyse kinematics of the BMX Supercross gate start , 2018, Sports biomechanics.

[29]  Geoffrey D Abrams,et al.  Biomechanical analysis of three tennis serve types using a markerless system , 2012, British Journal of Sports Medicine.

[30]  Xiaodong Yang,et al.  Effective 3D action recognition using EigenJoints , 2014, J. Vis. Commun. Image Represent..

[31]  Yogita M. Vaidya,et al.  Accuracy measurement of depth using Kinect sensor , 2016, 2016 Conference on Advances in Signal Processing (CASP).

[32]  Lei Wu,et al.  Effective Active Skeleton Representation for Low Latency Human Action Recognition , 2016, IEEE Transactions on Multimedia.

[33]  Luc Van Gool,et al.  3D reconstruction of freely moving persons for re-identification with a depth sensor , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[34]  M. M. Reijne,et al.  Accuracy of human motion capture systems for sport applications; state-of-the-art review , 2018, European journal of sport science.

[35]  Adso Fernández-Baena,et al.  Biomechanical Validation of Upper-Body and Lower-Body Joint Movements of Kinect Motion Capture Data for Rehabilitation Treatments , 2012, 2012 Fourth International Conference on Intelligent Networking and Collaborative Systems.

[36]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[37]  Petros Daras,et al.  Real-Time Skeleton-Tracking-Based Human Action Recognition Using Kinect Data , 2014, MMM.

[38]  B. Elliott,et al.  Long-axis rotation: The missing link in proximal-to-distal segmental sequencing , 2000, Journal of sports sciences.

[39]  Takahiro Okabe,et al.  Fast unsupervised ego-action learning for first-person sports videos , 2011, CVPR 2011.

[40]  Rama Chellappa,et al.  Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Won Jong Jeon,et al.  Spatio-temporal pyramid matching for sports videos , 2008, MIR '08.

[42]  Victor Hugo C. De Albuquerque,et al.  Dynamic Evaluation and Treatment of the Movement Amplitude Using Kinect Sensor , 2018, IEEE Access.

[43]  Ramchandra Manthalkar,et al.  Recognition of vision-based activities of daily living using linear predictive coding of histogram of directional derivative , 2019, J. Ambient Intell. Humaniz. Comput..

[44]  Zhengang Wei,et al.  Automatic analysis of complex athlete techniques in broadcast taekwondo video , 2017, Multimedia Tools and Applications.

[45]  Alex Mihailidis,et al.  Feature-level combination of skeleton joints and body parts for accurate aggressive and agitated behavior recognition , 2017, J. Ambient Intell. Humaniz. Comput..

[46]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[47]  Chiou-Shann Fuh,et al.  Segmenting Highly Articulated Video Objects with Weak-Prior Random Forests , 2006, ECCV.

[48]  Wanqing Li,et al.  Action recognition based on a bag of 3D points , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[49]  Masami Akai,et al.  Relationship between three-dimensional kinematics of knee and trunk motion during shuttle run cutting , 2011, Journal of sports sciences.

[50]  Amr Sharaf,et al.  Real-Time Multi-scale Action Detection from 3D Skeleton Data , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[51]  Qi Wu,et al.  A Post-Rectification Approach of Depth Images of Kinect v2 for 3D Reconstruction of Indoor Scenes , 2017, ISPRS Int. J. Geo Inf..