Computer vision for sports: Current applications and research topics

Abstract The world of sports intrinsically involves fast and accurate motion that is not only challenging for competitors to master, but can be difficult for coaches and trainers to analyze, and for audiences to follow. The nature of most sports means that monitoring by the use of sensors or other devices fixed to players or equipment is generally not possible. This provides a rich set of opportunities for the application of computer vision techniques to help the competitors, coaches and audience. This paper discusses a selection of current commercial applications that use computer vision for sports analysis, and highlights some of the topics that are currently being addressed in the research community. A summary of on-line datasets to support research in this area is included.

[1]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Jingchen Liu,et al.  Detecting and Tracking Sports Players with Random Forests and Context-Conditioned Motion Models , 2014 .

[3]  Ming Xu,et al.  Tracking the soccer ball using multiple fixed cameras , 2009, Comput. Vis. Image Underst..

[4]  Thomas B. Moeslund,et al.  Classification of Sports Types Using Thermal Imagery , 2014 .

[5]  Markus Gross,et al.  Geometry Reconstruction of Players for Novel-View Synthesis of Sports Broadcasts , 2014 .

[6]  Nicu Sebe,et al.  The S-HOCK dataset: Analyzing crowds at the stadium , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  William J. Christmas,et al.  Ball Tracking for Tennis Video Annotation , 2014 .

[8]  Karsten Müller,et al.  Soccer Jersey Number Recognition Using Convolutional Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[9]  Fei-Fei Li,et al.  What, where and who? Classifying events by scene and object recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  Pascal Fua,et al.  Tracking Interacting Objects Optimally Using Integer Programming , 2014, ECCV.

[11]  Mark Everingham,et al.  Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation , 2010, BMVC.

[12]  Yuichi Ohta,et al.  Robust trajectory estimation of soccer players by using two cameras , 2008, 2008 19th International Conference on Pattern Recognition.

[13]  R. Storey.  Teletrack: A special effect , 1984 .

[14]  Chien-Li Chou,et al.  Screen-strategy analysis in broadcast basketball video using player tracking , 2011, 2011 Visual Communications and Image Processing (VCIP).

[15]  Anthony R. Dick,et al.  AFL Player Detection and Tracking , 2015, 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[16]  Hua-Tsung Chen,et al.  Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video , 2009, J. Vis. Commun. Image Represent..

[17]  Juan Carlos Niebles,et al.  Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification , 2010, ECCV.

[18]  Christophe De Vleeschouwer,et al.  An Autonomous Framework to Produce and Distribute Personalized Team-Sport Video Summaries: A Basketball Case Study , 2011, IEEE Transactions on Multimedia.

[19]  Sukadev Meher,et al.  A real-time trajectory-based ball detection-and-tracking framework for basketball video , 2013 .

[20]  D. Radosav,et al.  AdaBoost in basketball player identification , 2012, 2012 IEEE 13th International Symposium on Computational Intelligence and Informatics (CINTI).

[21]  Alan Hanjalic,et al.  Adaptive extraction of highlights from a sport video based on excitement modeling , 2005, IEEE Transactions on Multimedia.

[22]  Mubarak Shah,et al.  Action MACH a spatio-temporal Maximum Average Correlation Height filter for action recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Ricardo M. L. Barros,et al.  Tracking soccer players aiming their kinematical motion analysis , 2006, Comput. Vis. Image Underst..

[24]  Thomas Mauthner,et al.  Indoor Activity Detection and Recognition for Automated Sport Games Analysis , 2014 .

[25]  Thomas B. Moeslund,et al.  Occupancy Analysis of Sports Arenas using Thermal Imaging , 2012, VISAPP.

[26]  Pascal Fua,et al.  Tracking Interacting Objects Using Intertwined Flows , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Hamid Abrishami Moghaddam,et al.  A survey on player tracking in soccer videos , 2017, Comput. Vis. Image Underst..

[28]  Graham Thomas,et al.  Image-based camera tracking for athletics , 2009, 2009 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting.

[29]  Matej Kristan,et al.  A trajectory-based analysis of coordinated team activity in a basketball game , 2009, Comput. Vis. Image Underst..

[30]  C. Krishna Mohan,et al.  Classification of sport videos using edge-based features and autoassociative neural network models , 2010, Signal Image Video Process..

[31]  Ales Leonardis,et al.  Multi-target tracking in team-sports videos via multi-level context-conditioned latent behaviour models , 2014, BMVC.

[32]  Tiziana D'Orazio,et al.  Real-time multiview analysis of soccer matches for understanding interactions between ball and players , 2008, CIVR '08.

[33]  Atsuo Yoshitaka,et al.  Soccer video summarization based on cinematography and motion analysis , 2014, 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP).

[34]  Umesh C. Pati,et al.  A novel algorithm for detection of soccer ball and player , 2015, 2015 International Conference on Communications and Signal Processing (ICCSP).

[35]  Chunru Wan,et al.  The application of edge feature in automatic sports genre classification , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[36]  Noel E. O'Connor,et al.  Real-Time Event Detection in Field Sport Videos , 2014 .

[37]  Sridha Sridharan,et al.  Representing Team Behaviours from Noisy Data Using Player Role , 2014, CVPR 2014.

[38]  Amir Roshan Zamir,et al.  Action Recognition in Realistic Sports Videos , 2014 .

[39]  Derek R. Magee,et al.  Computer vision based analysis in sport environments , 2009, Computer Vision and Image Understanding.

[40]  Oliver Grau,et al.  Use of 3D techniques for virtual production , 2000, IS&T/SPIE Electronic Imaging.

[41]  Mubarak Shah,et al.  Tracking When the Camera Looks Away , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[42]  Somnath Sengupta,et al.  Bayesian Network-Based Customized Highlight Generation for Broadcast Soccer Videos , 2015, IEEE Transactions on Broadcasting.

[43]  David Menotti,et al.  An overview of automatic event detection in soccer matches , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[44]  Narendra Ahuja,et al.  Robust multi-object tracking via cross-domain contextual information for sports video analysis , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[45]  Pascal Fua,et al.  What Players do with the Ball: A Physically Constrained Interaction Modeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  A. Murat Tekalp,et al.  Automatic soccer video analysis and summarization , 2003, IEEE Trans. Image Process..

[47]  Wen Gao,et al.  Jersey number detection in sports video for athlete identification , 2005, Visual Communications and Image Processing.

[48]  Clinton Fookes,et al.  Non-Invasive Performance Measurement in Combat Sports , 2016 .

[49]  Graham A. Thomas,et al.  Real-time camera tracking using sports pitch markings , 2007, Journal of Real-Time Image Processing.

[50]  C. Krishna Mohan,et al.  Event-Based Sports Videos Classification Using HMM Framework , 2014 .

[51]  Tiziana D'Orazio,et al.  A Semi-automatic System for Ground Truth Generation of Soccer Video Sequences , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[52]  Vito Renó,et al.  Tennis Player Segmentation for Semantic Behavior Analysis , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[53]  Matej Kristan,et al.  Closed-world tracking of multiple interacting targets for indoor-sports applications , 2009, Comput. Vis. Image Underst..

[54]  Xiaoming Liu,et al.  Sports Videos in the Wild (SVW): A video dataset for sports analysis , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[55]  S. Meher,et al.  Real-time position estimation and tracking of a basketball , 2012, 2012 IEEE International Conference on Signal Processing, Computing and Control.

[56]  Christophe De Vleeschouwer,et al.  Distributed video acquisition and annotation for sport-event summarization , 2008 .

[57]  Hideo Saito,et al.  Plane Approximation-Based Approach for 3D Reconstruction of Ball Trajectory for Performance Analysis in Table Tennis , 2014, CVPR 2014.

[58]  Josef Halbinger,et al.  Video-Based Soccer Ball Detection in Difficult Situations , 2013, icSPORTS 2013.

[59]  Ming Xu,et al.  Tracking football players with multiple cameras , 2004 .

[60]  Du Tran,et al.  Human Activity Recognition with Metric Learning , 2008, ECCV.

[61]  Sridha Sridharan,et al.  Predicting Ball Ownership in Basketball from a Monocular View Using Only Player Trajectories , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[62]  N. Owens,et al.  Hawk-eye tennis system , 2003 .

[63]  James J. Little,et al.  Identifying players in broadcast sports videos using conditional random fields , 2011, CVPR 2011.

[64]  P. Sanguansat,et al.  TF-RNF: A novel term weighting scheme for sports video classification , 2012, 2012 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2012).

[65]  Aboul Ella Hassanien,et al.  SVM-based soccer video summarization system , 2011, 2011 Third World Congress on Nature and Biologically Inspired Computing.

[66]  Carsten Griwodz,et al.  Soccer video and player position dataset , 2014, MMSys '14.

[67]  Lifang Wu,et al.  A New Method of Object Segmentation in the Basketball Videos , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[68]  Indriyati Atmosukarto,et al.  Recognizing Team Formation in American Football , 2014, CVPR 2014.

[69]  John H. L. Hansen,et al.  Automatic excitement-level detection for sports highlights generation , 2010, INTERSPEECH.

[70]  Jae Lee,et al.  Activity Identification Utilizing Data Mining Techniques , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).