Fast camera self-calibration for synthesizing Free Viewpoint soccer Video

Recently, non-fixed camera-based free viewpoint sports video synthesis has become very popular. Camera calibration is an indispensable step in free viewpoint video synthesis, and the calibration has to be done frame by frame for a non-fixed camera. Thus, calibration speed is of great significance in real-time application. In this paper, a fast self-calibration method for a non-fixed camera is proposed to estimate the homography matrix between a camera image and a soccer field model. As far as we know, it is the first time to propose constructing feature vectors by analyzing crossing points of field lines in both camera image and field model. Therefore, different from previous methods that evaluate all the possible homography matrices and select the best one, our proposed method only evaluates a small number of homography matrices based on the matching result of the constructed feature vectors. Experimental results show that the proposed method is much faster than other methods with only a slight loss of calibration accuracy that is negligible in final synthesized videos.

[1]  Peter H. N. de With,et al.  Fast camera calibration for the analysis of sport sequences , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[2]  Wolfgang Effelsberg,et al.  Robust camera calibration for sport videos using court models , 2003, IS&T/SPIE Electronic Imaging.

[3]  Toshiaki Fujii,et al.  Free-Viewpoint TV , 2011, IEEE Signal Processing Magazine.

[4]  Yuichi Ohta,et al.  Live 3D Video in Soccer Stadium , 2003, SIGGRAPH '03.

[5]  Frédo Durand,et al.  Billboard clouds for extreme model simplification , 2003, ACM Trans. Graph..

[6]  Vicent Caselles,et al.  Homography estimation using one ellipse correspondence and minimal additional information , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[7]  Adrian Hilton,et al.  A Free-Viewpoint Video System for Visualization of Sport Scenes , 2007 .

[8]  Anil K. Jain,et al.  Automatic classification of tennis video for high-level content-based retrieval , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[9]  Baljinder Singh Bal,et al.  Hawk Eye: A Logical Innovative Technology Use in Sports for Effective Decision Making , 2012 .

[10]  Thomas Wiegand,et al.  3D Video and Free Viewpoint Video - Technologies, Applications and MPEG Standards , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[11]  Adrian Hilton,et al.  A Comparative Study of Free-Viewpoint Video Techniques For sports events , 2006 .

[12]  Toshiaki Fujii,et al.  A practical implementation of free viewpoint video system for soccer games , 2015, Electronic Imaging.

[13]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[14]  Yonina C. Eldar,et al.  A probabilistic Hough transform , 1991, Pattern Recognit..

[15]  Hiroshi Sankoh,et al.  Accurate silhouette extraction of multiple moving objects for free viewpoint sports video synthesis , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[16]  Guannan Gao,et al.  Probabilistic Hough Transform , 2011 .

[17]  Hiroshi Sankoh,et al.  Robust moving camera calibration for synthesizing free viewpoint soccer video , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[18]  Hiroshi Sankoh,et al.  Automatic camera self-calibration for immersive navigation of free viewpoint sports video , 2016, 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP).

[19]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..