Estimating Player Positions from Padel High-Angle Videos: Accuracy Comparison of Recent Computer Vision Methods

The estimation of player positions is key for performance analysis in sport. In this paper, we focus on image-based, single-angle, player position estimation in padel. Unlike tennis, the primary camera view in professional padel videos follows a de facto standard, consisting of a high-angle shot at about 7.6 m above the court floor. This camera angle reduces the occlusion impact of the mesh that stands over the glass walls, and offers a convenient view for judging the depth of the ball and the player positions and poses. We evaluate and compare the accuracy of state-of-the-art computer vision methods on a large set of images from both amateur videos and publicly available videos from the major international padel circuit. The methods we analyze include object detection, image segmentation and pose estimation techniques, all of them based on deep convolutional neural networks. We report accuracy and average precision with respect to manually-annotated video frames. The best results are obtained by top-down pose estimation methods, which offer a detection rate of 99.8% and a RMSE below 5 and 12 cm for horizontal/vertical court-space coordinates (deviations from predicted and ground-truth player positions). These results demonstrate the suitability of pose estimation methods based on deep convolutional neural networks for estimating player positions from single-angle padel videos. Immediate applications of this work include the player and team analysis of the large collection of publicly available videos from international circuits, as well as an inexpensive method to get player positional data in amateur padel clubs.

[1]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[2]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Ricardo Machado Leite de Barros,et al.  Validation of a video-based system for automatic tracking of tennis players , 2018 .

[4]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Zhaoxiang Zhang,et al.  Sequence Level Semantics Aggregation for Video Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Toon De Pessemier,et al.  Badminton Activity Recognition Using Accelerometer Data , 2020, Sensors.

[8]  Stephen Lin,et al.  Deformable ConvNets V2: More Deformable, Better Results , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Avinash G. Keskar,et al.  Ball tracking in sports: a survey , 2019, Artificial Intelligence Review.

[10]  Nic James,et al.  The Effect of the Return of Serve on the Server Pair’s Movement Parameters and Rally Outcome in Padel Using Cluster Analysis , 2019, Front. Psychol..

[11]  Mustafa Gul,et al.  A Deep Learning and Computer Vision Based Multi-Player Tracker for Squash , 2020 .

[12]  Yujie Wang,et al.  Flow-Guided Feature Aggregation for Video Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Osami Yamamoto,et al.  Analyzing Movements of Tennis Players by Dynamic Image Processing , 2007 .

[14]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[15]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[16]  Armando Sousa,et al.  Survey on team tracking techniques applied to sports , 2010, 2010 International Conference on Autonomous and Intelligent Systems, AIS 2010.

[17]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Anil C. Kokaram,et al.  Joint audio visual retrieval for tennis broadcasts , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[19]  J. Guzmán,et al.  Stroke Analysis in Padel According to Match Outcome and Game Side on Court , 2020, International journal of environmental research and public health.

[20]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Fahad Shahbaz Khan,et al.  D2Det: Towards High Quality Object Detection and Instance Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Masahiro Shinya,et al.  A split-step shortens the time to perform a choice reaction step-and-reach movement in a simulated tennis task , 2009, Journal of sports sciences.

[24]  Maria Skublewska-Paszkowska,et al.  Learning Three Dimensional Tennis Shots Using Graph Convolutional Networks , 2020, Sensors.

[25]  Mao Ye,et al.  Distribution-Aware Coordinate Representation for Human Pose Estimation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yichen Wei,et al.  Simple Baselines for Human Pose Estimation and Tracking , 2018, ECCV.

[27]  Jinzi Mao,et al.  Tracking a tennis ball using image processing techniques , 2006 .

[28]  Kai Chen,et al.  Hybrid Task Cascade for Instance Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[32]  Kai Chen,et al.  MMDetection: Open MMLab Detection Toolbox and Benchmark , 2019, ArXiv.

[33]  Mercedes Sanchís Almenara,et al.  Padel: A Quantitative study of the shots and movements in the high-performance , 2013 .

[34]  Huang-Chia Shih,et al.  A Survey of Content-Aware Video Analysis for Sports , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  William J. Christmas,et al.  A Tennis Ball Tracking Algorithm for Automatic Annotation of Tennis Match , 2005, BMVC.

[37]  Nic James,et al.  Measurement error associated with the SAGIT/Squash computer tracking software , 2010 .

[38]  Yichen Wei,et al.  Deep Feature Flow for Video Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Javier Courel-Ibáñez,et al.  IMU gyroscopes are a valid alternative to 3D optical motion capture system for angular kinematics analysis in tennis , 2020 .

[40]  T. García-Calvo,et al.  A Longitudinal Exploration of Match Running Performance during a Football Match in the Spanish La Liga: A Four-Season Study † , 2021, International journal of environmental research and public health.

[41]  Guan Huang,et al.  The Devil Is in the Details: Delving Into Unbiased Data Processing for Human Pose Estimation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Yves Jean,et al.  Ball tracking and virtual replays for innovative tennis broadcasts , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[43]  Sam Robertson,et al.  Machine and deep learning for sport-specific movement recognition: a systematic review of model development and performance , 2018, Journal of sports sciences.

[44]  Martin Lames,et al.  Validation of electronic performance and tracking systems EPTS under field conditions , 2018, PloS one.

[45]  Toshio Asano,et al.  Quantitative Evaluation of Tennis Plays by Computer Vision , 2013 .

[46]  Cheng-Chih Wu,et al.  Learning Tennis through Video-based Reflective Learning by Using Motion-Tracking Sensors , 2020, J. Educ. Technol. Soc..

[47]  Anil K. Jain,et al.  Automatic classification of tennis video for high-level content-based retrieval , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[48]  Laura Leal-Taixé,et al.  Tracking Without Bells and Whistles , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[49]  Nuno Vasconcelos,et al.  Cascade R-CNN: High Quality Object Detection and Instance Segmentation , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Brejesh Lall,et al.  Automated ball tracking in tennis videos , 2015, 2015 Third International Conference on Image Information Processing (ICIIP).

[51]  Gerardo Hermosillo,et al.  Learning From Crowds , 2010, J. Mach. Learn. Res..