Predicting Shot Locations in Tennis Using Spatiotemporal Data

Over the past decade, vision-based tracking systems have been successfully deployed in professional sports such as tennis and cricket for enhanced broadcast visualizations as well as aiding umpiring decisions. Despite the high-level of accuracy of the tracking systems and the sheer volume of spatiotemporal data they generate, the use of this high quality data for quantitative player performance and prediction has been lacking. In this paper, we present a method which predicts the location of a future shot based on the spatiotemporal parameters of the incoming shots (i.e. shot speed, location, angle and feet location) from such a vision system. Having the ability to accurately predict future short-term events has enormous implications in the area of automatic sports broadcasting in addition to coaching and commentary domains. Using Hawk-Eye data from the 2012 Australian Open Men's draw, we utilize a Dynamic Bayesian Network to model player behaviors and use an online model adaptation method to match the player's behavior to enhance shot predictability. To show the utility of our approach, we analyze the shot predictability of the top 3 players seeds in the tournament (Djokovic, Federer and Nadal) as they played the most amounts of games.

[1]  Yu-Han Chang,et al.  Deconstructing the Rebound with Optical Tracking Data , 2012 .

[2]  Robert T. Collins,et al.  Multitarget data association with higher-order motion models , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Pascal Fua,et al.  Take your eyes off the ball: Improving ball-tracking by focusing on team play , 2014, Comput. Vis. Image Underst..

[5]  Indriyati Atmosukarto,et al.  Automatic Recognition of Offensive Team Formation in American Football Plays , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[7]  Peter Carr,et al.  Characterizing Multi-Agent Team Behavior from Partial Team Tracings: Evidence from the English Premier League , 2012, AAAI.

[8]  Siddhartha S. Srinivasa,et al.  Planning-based prediction for pedestrians , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Silvio Savarese,et al.  A Unified Framework for Multi-target Tracking and Collective Activity Recognition , 2012, ECCV.

[10]  Erik G. Learned-Miller,et al.  Online domain adaptation of a pre-trained cascade of classifiers , 2011, CVPR 2011.

[11]  Sridha Sridharan,et al.  Recognising Team Activities from Noisy Data , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Irfan A. Essa,et al.  Motion fields to predict play evolution in dynamic sport scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Ivan Laptev,et al.  Data-driven crowd analysis in videos , 2011, ICCV.

[14]  Trevor Darrell,et al.  What you saw is not what you get: Domain adaptation using asymmetric kernel transforms , 2011, CVPR 2011.

[15]  Alan Fern,et al.  An Application of Transfer to American Football: From Observation of Raw Video to Control in a Simulated Environment , 2011, AI Mag..

[16]  Rama Chellappa,et al.  Group motion segmentation using a Spatio-Temporal Driving Force Model , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[18]  Yamada Makoto,et al.  No Bias Left Behind: Covariate Shift Adaptation for Discriminative 3D Pose Estimation , 2012 .

[19]  Wen Gao,et al.  Trajectory based event tactics analysis in broadcast sports video , 2007, ACM Multimedia.

[20]  Ivor W. Tsang,et al.  Domain Transfer SVM for video concept detection , 2009, CVPR 2009.

[21]  Matej Kristan,et al.  A trajectory-based analysis of coordinated team activity in a basketball game , 2009, Comput. Vis. Image Underst..

[22]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[23]  Rama Chellappa,et al.  Learning multi-modal densities on Discriminative Temporal Interaction Manifold for group activity recognition , 2009, CVPR.

[24]  P. Lucey,et al.  “ Sweet-Spot ” : Using Spatiotemporal Data to Discover and Predict Shots in Tennis , 2013 .

[25]  Charless C. Fowlkes,et al.  Globally-optimal greedy algorithms for tracking a variable number of objects , 2011, CVPR 2011.

[26]  Patrick Bouthemy,et al.  Understanding Sports Video Using Players Trajectories , 2011, Intelligent Video Event Analysis and Understanding.

[27]  Fernando De la Torre,et al.  Max-Margin Early Event Detectors , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Aaron F. Bobick,et al.  A Framework for Recognizing Multi-Agent Action from Visual Evidence , 1999, AAAI/IAAI.

[29]  Rong Yan,et al.  Cross-domain video concept detection using adaptive svms , 2007, ACM Multimedia.

[30]  Peter Carr,et al.  Assessing team strategy using spatiotemporal data , 2013, KDD.

[31]  Ling Shao,et al.  Intelligent Video Event Analysis and Understanding , 2010, Intelligent Video Event Analysis and Understanding.

[32]  Michael S. Ryoo,et al.  Human activity prediction: Early recognition of ongoing activities from streaming videos , 2011, 2011 International Conference on Computer Vision.