Ball tracking in sports: a survey

Increase in the number of sport lovers in games like football, cricket, etc. has created a need for digging, analyzing and presenting more and more multidimensional information to them. Different classes of people require different kinds of information and this expands the space and scale of the required information. Tracking of ball movement is of utmost importance for extracting any information from the ball based sports video sequences. Based on the literature survey, we have initially proposed a block diagram depicting different steps and flow of a general tracking process. The paper further follows the same flow throughout. Detection is the first step of tracking. Dynamic and unpredictable nature of ball appearance, movement and continuously changing background make the detection and tracking processes challenging. Due to these challenges, many researchers have been attracted to this problem and have produced good results under specific conditions. However, generalization of the published work and algorithms to different sports is a distant dream. This paper is an effort to present an exhaustive survey of all the published research works on ball tracking in a categorical manner. The work also reviews the used techniques, their performance, advantages, limitations and their suitability for a particular sport. Finally, we present discussions on the published work so far and our views and opinions followed by a modified block diagram of the tracking process. The paper concludes with the final observations and suggestions on scope of future work.

[1]  Min-Chun Hu,et al.  Robust Camera Calibration and Player Tracking in Broadcast Basketball Video , 2011, IEEE Transactions on Multimedia.

[2]  Susanna Nilsson.  Interaction Without Gesture or Speech -- A Gaze Controlled AR System , 2007 .

[3]  Ming Xu,et al.  Strathprints Institutional Repository (2008) Real-time Modeling of 3-d Soccer Ball Trajectories from Multiple Fixed Cameras. Ieee Transactions on Circuits and Systems for Video Technology, 18 (3). Pp. 350-362. Issn 1051-8215 , 2022 .

[4]  Yongduek Seo,et al.  Probabilistic Tracking of the Soccer Ball , 2004, ECCV Workshop SMVP.

[5]  L. S. Dooley,et al.  High-motion table tennis ball tracking for umpiring applications , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[6]  Loong Fah Cheong,et al.  Automatic camera calibration of broadcast tennis video with applications to 3D virtual content insertion and ball detection and tracking , 2009, Comput. Vis. Image Underst..

[7]  Loong Fah Cheong,et al.  A trajectory-based ball detection and tracking algorithm in broadcast tennis video , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[8]  Peter H. N. de With,et al.  Fast camera calibration for the analysis of sport sequences , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[9]  Jia Liu,et al.  Automatic player labeling, tracking and field registration and trajectory mapping in broadcast soccer video , 2011, TIST.

[10]  Anil Kokaram,et al.  Content Controlled Image Representation for Sports Streaming , 2005 .

[11]  Qi Tian,et al.  A robust and accumulator-free ellipse hough transform , 2004, MULTIMEDIA '04.

[12]  Vincent Lepetit,et al.  Robust data association for online application , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[13]  Qi Tian,et al.  Trajectory-Based Ball Detection and Tracking in Broadcast Soccer Video , 2006, IEEE Transactions on Multimedia.

[14]  S. Meher,et al.  A trajectory-based ball detection and tracking system with applications to shot-type identification in volleyball videos , 2012, 2012 International Conference on Signal Processing and Communications (SPCOM).

[15]  S. Meher,et al.  Real-time position estimation and tracking of a basketball , 2012, 2012 IEEE International Conference on Signal Processing, Computing and Control.

[16]  Qi Tian,et al.  Estimation of the ball size in broadcast soccer video using salient objects , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[17]  Wolfgang Effelsberg,et al.  Robust camera calibration for sport videos using court models , 2003, IS&T/SPIE Electronic Imaging.

[18]  Qi Tian,et al.  Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video , 2003, MULTIMEDIA '03.

[19]  Uday B. Desai,et al.  Small Object Detection and Tracking: Algorithm, Analysis and Application , 2005, PReMI.

[20]  Tetsuya Takiguchi,et al.  Digital camera work for soccer video production with event recognition and accurate ball tracking by switching search method , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[21]  Abir El Abed,et al.  Comparison of Statistical and Shape-Based Approaches for Non-rigid Motion Tracking with Missing Data Using a Particle Filter , 2006, ACIVS.

[22]  Anil C. Kokaram,et al.  Semantic Event Detection in Sports Through Motion Understanding , 2004, CIVR.

[23]  A. Singhal,et al.  Tracking soccer players using the graph representation , 2008 .

[24]  Wen Gao,et al.  Event Tactic Analysis Based on Broadcast Sports Video , 2009, IEEE Trans. Multim..

[25]  Pascal Fua,et al.  Tracking Interacting Objects Using Intertwined Flows , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  René van de Molengraft,et al.  Ball localization and tracking in a highly dynamic table soccer environment , 2012 .

[27]  Leslie Pack Kaelbling,et al.  Tracking the spin on a ping pong ball with the quaternion Bingham filter , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Yuichi Ohta,et al.  Real-time soccer player tracking method by utilizing shadow regions , 2010, ACM Multimedia.

[29]  Hubert P. H. Shum,et al.  A spatiotemporal approach to extract the 3D trajectory of the baseball from a single view video sequence , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[30]  Tiziana D'Orazio,et al.  An Investigation Into the Feasibility of Real-Time Soccer Offside Detection From a Multiple Camera System , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Yongduek Seo,et al.  Tracking Soccer Ball in TV Broadcast Video , 2005, ICIAP.

[32]  Oliver Grau,et al.  Tracking football player movement from a single moving camera using particle filters , 2006 .

[33]  Xinguo Yu,et al.  3D reconstruction and enrichment of broadcast soccer video , 2004, MULTIMEDIA '04.

[34]  J. Kittler,et al.  Improving the accuracy of automatic tennis video annotation by high level grammar , 2007 .

[35]  Hideo Saito,et al.  AR Display of Visual Aids for Supporting Pool Games by Online Markerless Tracking , 2007 .

[36]  Berthold Bäuml,et al.  Realtime perception for catching a flying ball with a mobile humanoid , 2011, 2011 IEEE International Conference on Robotics and Automation.

[37]  Kun Zhang,et al.  Improved high-speed vision system for table tennis robot , 2014, 2014 IEEE International Conference on Mechatronics and Automation.

[38]  Li Gao,et al.  A Real Time Ball Detection Framework For Soccer Video , 2009, 2009 16th International Conference on Systems, Signals and Image Processing.

[39]  Sukadev Meher,et al.  A trajectory-based ball detection and tracking system with applications to shooting angle and velocity estimation in basketball videos , 2013, 2013 Annual IEEE India Conference (INDICON).

[40]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[41]  Shihong Lao,et al.  Multiple Player Tracking in Sports Video: A Dual-Mode Two-Way Bayesian Inference Approach With Progressive Observation Modeling , 2011, IEEE Transactions on Image Processing.

[42]  Arit Thammano,et al.  Players tracking and ball detection for an automatic tennis video annotation , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[43]  Jean-Marc Odobez,et al.  Robust playfield segmentation using MAP adaptation , 2004, ICPR 2004.

[44]  Qiang Huang,et al.  A two layered data association approach for ball tracking , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  Uday B. Desai,et al.  Small and fast moving object detection and tracking in sports video sequences , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[46]  William J. Christmas,et al.  Ball Tracking for Tennis Video Annotation , 2014 .

[47]  Yongduek Seo,et al.  Where Are the Ball and Players? Soccer Game Analysis with Color Based Tracking and Image Mosaick , 1997, ICIAP.

[48]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[49]  Hideo Saito,et al.  Tracking soccer players based on homography among multiple views , 2003, Visual Communications and Image Processing.

[50]  Hua-Tsung Chen,et al.  Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video , 2009, J. Vis. Commun. Image Represent..

[51]  Yu-Jin Zhang,et al.  Tracking Ball and Players with Applications to Highlight Ranking of Broadcasting Table Tennis Video , 2006, The Proceedings of the Multiconference on "Computational Engineering in Systems Applications".

[52]  Yongduek Seo,et al.  Physics-based 3D position analysis of a soccer ball from monocular image sequences , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[53]  Kristine L. Bell,et al.  A Tutorial on Particle Filters for Online Nonlinear/NonGaussian Bayesian Tracking , 2007 .

[54]  Pascal Fua,et al.  Tracking Interacting Objects Optimally Using Integer Programming , 2014, ECCV.

[55]  Tiziana D'Orazio,et al.  A visual system for real time detection of goal events during soccer matches , 2009, Comput. Vis. Image Underst..

[56]  Wei Wei,et al.  A tracking and predicting scheme for ping pong robot , 2011, Journal of Zhejiang University SCIENCE C.

[57]  Changsheng Xu,et al.  Event tactic analysis based on player and ball trajectory in broadcast video , 2008, CIVR '08.

[58]  Tiziana D'Orazio,et al.  Real-time multiview analysis of soccer matches for understanding interactions between ball and players , 2008, CIVR '08.

[59]  Jinchang Ren,et al.  Generating ball trajectory in soccer video sequences , 2006 .

[60]  Yoshiaki Shirai,et al.  Tracking players and estimation of the 3D position of a ball in soccer games , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[61]  Yoshiaki Shirai,et al.  Estimation of Ball Route under Overlapping with Players and Lines in Soccer Video Image Sequence , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[62]  Yoshiaki Shirai,et al.  Tracking players and a ball in soccer games , 1999, Proceedings. 1999 IEEE/SICE/RSJ. International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI'99 (Cat. No.99TH8480).

[63]  Ming Xu,et al.  A general framework for 3D soccer ball estimation and tracking , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[64]  Y. Shirai,et al.  Ball Route Estimation in Broadcast Soccer Video , 2006 .

[65]  Baris David Ekinci,et al.  A ball tracking system for offline tennis videos , 2008 .

[66]  Shamik Sural,et al.  Ball detection from broadcast soccer videos using static and dynamic features , 2008, J. Vis. Commun. Image Represent..

[67]  Ja-Ling Wu,et al.  WOW: wild-open warning for broadcast basketball video based on player trajectory , 2009, MM '09.

[68]  Yoshiaki Shirai,et al.  Tracking players and a ball in video image sequence and estimating camera parameters for 3D interpretation of soccer games , 2002, Object recognition supported by user interaction for service robots.

[69]  Chng Eng Siong,et al.  A Player-Possession Acquisition System for Broadcast Soccer Video , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[70]  Hua-Tsung Chen,et al.  A Trajectory-Based Ball Tracking Framework with Visual Enrichment for Broadcast Baseball Videos , 2008, J. Inf. Sci. Eng..

[71]  Ming Xu,et al.  Tracking the soccer ball using multiple fixed cameras , 2009, Comput. Vis. Image Underst..

[72]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[73]  Noel E. O'Connor,et al.  3D Estimation and Visualization of Motion in a Multicamera Network for Sports , 2011, 2011 Irish Machine Vision and Image Processing Conference.

[74]  Sukadev Meher,et al.  A real-time trajectory-based ball detection-and-tracking framework for basketball video , 2013 .

[75]  Xinguo Yu,et al.  Trajectory-Based Ball Detection and Tracking in Broadcast Soccer Video with the Aid of Camera Motion Recovery , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[76]  Nicola Ancona,et al.  Ball detection in static images with Support Vector Machines for classification , 2003, Image Vis. Comput..

[77]  Qiang Huang,et al.  Tennis Ball Tracking Using a Two-Layered Data Association Approach , 2015, IEEE Transactions on Multimedia.

[78]  Hua-Tsung Chen,et al.  Ball tracking and 3D trajectory approximation with applications to tactics analysis from single-camera volleyball sequences , 2012, Multimedia Tools and Applications.

[79]  Wen Gao,et al.  Extracting 3D information from broadcast soccer video , 2006, Image Vis. Comput..

[80]  Vincenzo Lippiello,et al.  Robotic Ball Catching with an Eye-in-Hand Single-Camera System , 2015, IEEE Transactions on Control Systems Technology.

[81]  Cataldo Guaragnella,et al.  A new algorithm for ball recognition using circle Hough transform and neural classifier , 2004, Pattern Recognit..

[82]  Yan Li,et al.  Evaluating the performance of systems for tracking football players and ball , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[83]  Michael Beetz,et al.  Rao-Blackwellized Resampling Particle Filter for Real-time Player Tracking in Sports , 2009, VISAPP.

[84]  Goran Vu.  Physics-Based Modelling of Human Motion using Kalman Filter and Collision Avoidance Algorithm , 2005 .

[85]  Yves Jean,et al.  Instantly indexed multimedia databases of real world events , 2002, IEEE Trans. Multim..

[86]  Pascal Fua,et al.  Take your eyes off the ball: Improving ball-tracking by focusing on team play , 2014, Comput. Vis. Image Underst..

[87]  Ian D. Reid,et al.  3D Trajectories from a Single Viewpoint using Shadows , 1998, BMVC.

[88]  Hubert P. H. Shum,et al.  Tracking the translational and rotational movement of the ball using high-speed camera movies , 2005, IEEE International Conference on Image Processing 2005.

[89]  J. Pers,et al.  Physics-based modelling of human motion using Kalman filter and collision avoidance algorithm , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[90]  Yu Huang,et al.  A method of small object detection and tracking based on particle filters , 2008, 2008 19th International Conference on Pattern Recognition.

[91]  Qi Tian,et al.  A novel ball detection framework for real soccer video , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[92]  Reza Safabakhsh,et al.  Effective tracking of the players and ball in indoor soccer games in the presence of occlusion , 2009, 2009 14th International CSI Computer Conference.

[93]  Yihong Gong,et al.  Feature design in soccer video indexing , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[94]  Petros Daras,et al.  A low-cost performance analysis and coaching system for tennis , 2010 .

[95]  Yongduek Seo,et al.  Automatic initialization for 3D soccer player tracking , 2011, Pattern Recognit. Lett..

[96]  Pascal Fua,et al.  What Players do with the Ball: A Physically Constrained Interaction Modeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[97]  Jinchang Ren,et al.  Real-time 3D Soccer Ball Tracking from Multiple Cameras , 2004 .

[98]  Chao Lan,et al.  Anomaly Detection , 2018, Encyclopedia of GIS.

[99]  Yoshiaki Shirai,et al.  Ball route estimation under heavy occlusion in broadcast soccer video , 2009, Comput. Vis. Image Underst..

[100]  Martin Buss,et al.  Robot basketball: A comparison of ball dribbling with visual and force/torque feedback , 2009, 2009 IEEE International Conference on Robotics and Automation.

[101]  Zhiqiang Wang,et al.  A statistical method for analysis of technical data of a badminton match based on 2-D seriate images , 2007 .

[102]  Jules-Raymond Tapamo,et al.  Soccer video analysis by ball, player and referee tracking , 2006 .

[103]  Qingshan Liu,et al.  An effective and fast soccer ball detection and tracking method , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[104]  William J. Christmas,et al.  A Tennis Ball Tracking Algorithm for Automatic Annotation of Tennis Match , 2005, BMVC.

[105]  Qi Tian,et al.  A ball tracking framework for broadcast soccer video , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[106]  Sridha Sridharan,et al.  Forecasting Events Using an Augmented Hidden Conditional Random Field , 2014, ACCV.

[107]  Livier Reithler,et al.  Physics based 3D ball tracking for tennis videos , 2010, 2010 International Workshop on Content Based Multimedia Indexing (CBMI).

[108]  David Windridge,et al.  Improved detection of ball hit events in a tennis game using multimodal information , 2011, AVSP.

[109]  Wen Gao,et al.  A Scheme for Ball Detection and Tracking in Broadcast Soccer Video , 2005, PCM.

[110]  David Windridge,et al.  Anomaly Detection and Knowledge Transfer in Automatic Sports Video Annotation , 2012, Detection and Identification of Rare Audiovisual Cues.

[111]  TaeYong Kim,et al.  Soccer Ball Tracking Using Dynamic Kalman Filter with Velocity Control , 2009, 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization.

[112]  Takeshi Ikenaga,et al.  Anti-occlusion observation model and automatic recovery for multi-view ball tracking in sports analysis , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[113]  Martin Buss,et al.  Robot Basketball: A Comparison of Ball Dribbling with Visual and Force/Torque Information , 2009, ICRA 2009.

[114]  Wen Gao,et al.  Video2Cartoon: A System for Converting Broadcast Soccer Video into 3D Cartoon Animation , 2007, IEEE Transactions on Consumer Electronics.

[115]  Hisashi Miyamori,et al.  Video annotation for content-based retrieval using human behavior analysis and domain knowledge , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[116]  Yves Jean,et al.  Ball tracking and virtual replays for innovative tennis broadcasts , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[117]  William J. Christmas,et al.  A Novel Data Association Algorithm for Object Tracking in Clutter with Application to Tennis Video Analysis , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[118]  Junqing Yu,et al.  Playfield and Ball Detection in Soccer Video , 2007, ISVC.

[119]  M. Kamarei,et al.  Ball Detection with the Aim of Corner Event Detection in Soccer Video , 2011, 2011 IEEE Ninth International Symposium on Parallel and Distributed Processing with Applications Workshops.

[120]  Boon-Lock Yeo,et al.  Analysis And Presentation Of Soccer Highlights From Digital Video , 1995 .

[121]  Nobuyuki Yagi,et al.  Distributed Particle Filtering for Multiocular Soccer-Ball Tracking , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[122]  Armando Sousa,et al.  Real Time Colour Based Player Tracking in Indoor Sports , 2011 .

[123]  Thomas Mauthner,et al.  Visual Tracking of Athletes in Beach Volleyball Using a Single Camera , 2007, Int. J. Comput. Sci. Sport.

[124]  Takeshi Ikenaga,et al.  Particle Filter with Ball Size Adaptive Tracking Window and Ball Feature Likelihood Model for Ball's 3D Position Tracking in Volleyball Analysis , 2015, PCM.

[125]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[126]  Yuichi Ohta,et al.  3D Tracking of a Soccer Ball Using Two Synchronized Cameras , 2007, PCM.

[127]  Qiang Huang,et al.  Detection of ball hits in a tennis game using audio and visual information , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[128]  Balasubramanian Raman,et al.  Automated Stroke Classification in Tennis , 2007, ICIAR.

[129]  Hua-Tsung Chen,et al.  Physics-Based Ball Tracking in Volleyball Videos with its Applications to Set Type Recognition and Action Detection , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[130]  Michael Beetz,et al.  ASPOGAMO: Automated Sports Games Analysis Models , 2009, Int. J. Comput. Sci. Sport.

[131]  David Windridge,et al.  A Memory Architecture and Contextual Reasoning Framework for Cognitive Vision , 2005, SCIA.

[132]  Oliver Birbach,et al.  A Multiple Hypothesis Approach for a Ball Tracking System , 2009, ICVS.

[133]  Hanqing Lu,et al.  An effective and fast soccer ball detection and tracking method , 2004, ICPR 2004.

[134]  Ming Xu,et al.  Real-time 3D Football Ball Tracking from Multiple Cameras , 2004, BMVC.

[135]  William J. Christmas,et al.  A Maximum A Posteriori Probability Viterbi Data Association Algorithm for Ball Tracking in Sports Video , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[136]  De Xu,et al.  Visual Measurement and Prediction of Ball Trajectory for Table Tennis Robot , 2010, IEEE Transactions on Instrumentation and Measurement.

[137]  Noel E. O'Connor,et al.  TennisSense: A platform for extracting semantic information from multi-camera tennis data , 2009, 2009 16th International Conference on Digital Signal Processing.

[138]  William J. Christmas,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Layered Data Association Using Graph-theoretic Formulation with Application to Tennis Ball Tracking in Monocular Sequences , 2022 .