As Seen on TV: Automatic Basketball Video Production using Gaussian-based Actionness and Game States Recognition

Automatic video production of sports aims at producing an aesthetic broadcast of sporting events. We present a new video system able to automatically produce a smooth and pleasant broadcast of Basketball games using a single fixed 4K camera. The system automatically detects and localizes players, ball and referees, to recognize main action coordinates and game states yielding to a professional cameraman-like production of the basketball event. We also release a fully annotated dataset consisting of single 4K camera and twelve-camera videos of basketball games.

[1]  Daiichiro Kato,et al.  Automatic tracking sensor camera system , 2001, IS&T/SPIE Electronic Imaging.

[2]  Yaser Sheikh,et al.  One-man-band: a touch screen interface for producing live multi-camera sports broadcasts , 2013, MM '13.

[3]  James H. Elder,et al.  Keep Your Eye on the Puck: Automatic Hockey Videography , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[4]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[5]  Yasuo Ariki,et al.  A Method of Digital Camera Work Focused on Players and a Ball: - Toward Automatic Contents Production System of Commentary Soccer Video by Digital Shooting , 2004, PCM.

[6]  Hironobu Fujiyoshi,et al.  Virtual camerawork for generating lecture video from high resolution images , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[7]  Kamalakar Karlapalem,et al.  Aesthetic Guideline Driven Photography by Robots , 2011, IJCAI.

[8]  Daiichiro Kato,et al.  Measurement method of zooming by a cameraman , 1998, Defense, Security, and Sensing.

[9]  Subramanian Ramanathan,et al.  Watch to Edit: Video Retargeting using Gaze , 2018, Comput. Graph. Forum.

[10]  Xiangyu Zhang,et al.  CrowdHuman: A Benchmark for Detecting Human in a Crowd , 2018, ArXiv.

[11]  Michael Gleicher,et al.  Re-cinematography: Improving the camerawork of casual video , 2008, TOMCCAP.

[12]  Chng Eng Siong,et al.  Automatic composition of broadcast sports video , 2008, Multimedia Systems.

[13]  Takao Tsuda,et al.  A High-Accuracy Image Composition System Using a Mobile Robotic Camera , 2008 .

[14]  Andrea Cavallaro,et al.  Multi-camera Scheduling for Video Production , 2011, 2011 Conference for Visual Media Production.

[15]  Jim Owens.  Television Sports Production , 2015 .

[16]  Peter Carr,et al.  Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording , 2013, ACM Multimedia.

[17]  Jianhui Chen,et al.  Mimicking Human Camera Operators , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[18]  Adrian Hilton,et al.  Computer vision for sports: Current applications and research topics , 2017, Comput. Vis. Image Underst..

[19]  James J. Little,et al.  Learning Online Smooth Predictors for Realtime Camera Planning Using Recurrent Decision Trees , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Michael Gleicher,et al.  Re-cinematography: improving the camera dynamics of casual video , 2007, ACM Multimedia.

[21]  N. Ohnishi,et al.  Soccer image sequence computed by a virtual camera , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[22]  Jianhui Chen,et al.  Autonomous Camera Systems: A Survey , 2014, WICED@AAAI.

[23]  Michael Gleicher,et al.  Virtual videography , 2007, TOMCCAP.

[24]  Christophe De Vleeschouwer,et al.  Personalized production of basketball videos from multi-sensored data under limited display resolution , 2010, Comput. Vis. Image Underst..

[25]  Li Fei-Fei,et al.  Detecting Events and Key Actors in Multi-person Videos , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Juliane Freud.  Shot By Shot A Practical Guide To Filmmaking , 2016 .

[27]  Ragnhild Eg,et al.  The Cameraman Operating My Virtual Camera is Artificial , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[28]  Sridha Sridharan,et al.  Recognising Team Activities from Noisy Data , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[29]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[31]  Tao Mei,et al.  Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Yasuo Ariki,et al.  Automatic Production System of Soccer Sports Video by Digital Camera Work Based on Situation Recognition , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[33]  Daiichiro Kato,et al.  Automatic control of a robot camera for broadcasting and subjective evaluation and analysis of reproduced images , 2000, Electronic Imaging.

[34]  E. Margolis,et al.  The SAGE Handbook of Visual Research Methods , 2011 .