Keep Your Eye on the Puck: Automatic Hockey Videography

While hockey involves a large playing surface, instantaneous play is typically localized to a smaller region of the ice. Live spectators thus attentively shift their gaze to follow play, and professional sports videographers pan and tilt their cameras to mimic this process. Unfortunately, manual videography is economically prohibitive below the elite level. Here we propose a system for automatically tracking play, allowing a high-definition video feed to be dynamically cropped and retargeted to a spectator's display device. We employ the puck as an objective surrogate for the location of play, and develop a novel method for ground-truthing puck location from high-definition video. This allows us to train a deep network regressor that uses the video imagery, optic flow, estimated player positions and team affiliation to predict the location of play. We show that our algorithm outperforms a simple 'follow the herd' strategy and results in a practical system for delivering high-quality curated video of amateur-level hockey games to remote spectators.

[1]  Shinji Ozawa,et al.  Automatic pan control system for broadcasting ball games based on audience's face direction , 2004, MULTIMEDIA '04.

[2]  Yasuo Ariki,et al.  Automatic Production System of Soccer Sports Video by Digital Camera Work Based on Situation Recognition , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[3]  Irfan A. Essa,et al.  Detecting regions of interest in dynamic scenes with camera motions , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Sukadev Meher,et al.  A real-time trajectory-based ball detection-and-tracking framework for basketball video , 2013 .

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Oliver Grau,et al.  Free-Viewpoint Video for TV Sport Production , 2010, Image and Geometry Processing for 3-D Cinematography.

[7]  Tiziana D'Orazio,et al.  Real-time multiview analysis of soccer matches for understanding interactions between ball and players , 2008, CIVR '08.

[8]  Irfan A. Essa,et al.  Motion fields to predict play evolution in dynamic sport scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Pascal Fua,et al.  Tracking Interacting Objects Using Intertwined Flows , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jianhui Chen,et al.  Mimicking Human Camera Operators , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[11]  Adrian Hilton,et al.  Computer vision for sports: Current applications and research topics , 2017, Comput. Vis. Image Underst..

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Hua-Tsung Chen,et al.  Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video , 2009, J. Vis. Commun. Image Represent..

[14]  E. Rossi,et al.  The relationship between visual resolution and cone spacing in the human fovea , 2009, Nature Neuroscience.

[15]  William J. Christmas,et al.  Ball Tracking for Tennis Video Annotation , 2014 .

[16]  Pascal Fua,et al.  Tracking Interacting Objects Optimally Using Integer Programming , 2014, ECCV.

[17]  Ming Xu,et al.  Tracking the soccer ball using multiple fixed cameras , 2009, Comput. Vis. Image Underst..

[18]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[19]  Ragnhild Eg,et al.  The Cameraman Operating My Virtual Camera is Artificial , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[20]  Sridha Sridharan,et al.  Predicting Ball Ownership in Basketball from a Monocular View Using Only Player Trajectories , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[21]  Nasser Kehtarnavaz,et al.  Ice-hockey puck detection and tracking for video highlighting , 2016, Signal Image Video Process..

[22]  Yanxi Liu,et al.  Tracking Sports Players with Context-Conditioned Motion Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Kathryn Bonnen,et al.  Dynamic mechanisms of visually guided 3D motion tracking. , 2017, Journal of neurophysiology.

[25]  Pascal Fua,et al.  What Players do with the Ball: A Physically Constrained Interaction Modeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Jitendra Malik,et al.  What will Happen Next? Forecasting Player Moves in Sports Videos , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  James J. Little,et al.  Learning Online Smooth Predictors for Realtime Camera Planning Using Recurrent Decision Trees , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).