PTZ camera network calibration from moving people in sports broadcasts

In sports broadcasts, networks consisting of pan-tilt-zoom (PTZ) cameras usually exhibit very wide baselines, making standard matching techniques for camera calibration very hard to apply. If, additionally, there is a lack of texture, finding corresponding image regions becomes almost impossible. However, such networks are often set up to observe dynamic scenes on a ground plane. Corresponding image trajectories produced by moving objects need to fulfill specific geometric constraints, which can be leveraged for camera calibration. We present a method which combines image trajectory matching with the self-calibration of rotating and zooming cameras, effectively reducing the remaining degrees of freedom in the matching stage to a 2D similarity transformation. Additionally, lines on the ground plane are used to improve the calibration. In the end, all extrinsic and intrinsic camera parameters are refined in a final bundle adjustment. The proposed algorithm was evaluated both qualitatively and quantitatively on four different soccer sequences.

[1]  Gideon P. Stein,et al.  Tracking from multiple view points: Self-calibration of space and time , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Ramakant Nevatia,et al.  Camera calibration from video of a walking human , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Ian D. Reid,et al.  Self-Calibration of Rotating and Zooming Cameras , 2002, International Journal of Computer Vision.

[7]  Richard I. Hartley,et al.  Self-Calibration of Stationary Cameras , 1997, International Journal of Computer Vision.

[8]  Paulo R. S. Mendonça,et al.  Bayesian autocalibration for surveillance , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  S. Shankar Sastry,et al.  Automatic Camera Network Localization using Object Image Tracks , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[11]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[12]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[13]  Marc Pollefeys,et al.  A Minimal Case Solution to the Calibrated Relative Pose Problem for the Case of Two Known Orientation Angles , 2010, ECCV.

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  W. Grimson,et al.  Ground Plane Rectification by Tracking Moving Objects , 2003 .

[16]  Marc Pollefeys,et al.  Robust multi-view camera calibration for wide-baseline camera networks , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[17]  YANQING CHEN,et al.  Algorithm 8 xx : CHOLMOD , supernodal sparse Cholesky factorization and update / downdate ∗ , 2006 .

[18]  Alberto Del Bimbo,et al.  Accurate self-calibration of two cameras by observations of a moving person on a ground plane , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[19]  Wolfgang Effelsberg,et al.  Robust camera calibration for sport videos using court models , 2003, IS&T/SPIE Electronic Imaging.

[20]  C. Jaynes.  Multi-view calibration from planar motion for video surveillance , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[21]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[22]  Manolis I. A. Lourakis.  Sparse Non-linear Least Squares Optimization for Geometric Vision , 2010, ECCV.

[23]  Graham A. Thomas,et al.  Real-time camera tracking using sports pitch markings , 2007, Journal of Real-Time Image Processing.

[24]  Patrick H. Worley,et al.  Algorithm 888: Spherical Harmonic Transform Algorithms , 2008, TOMS.

[25]  Richard I. Hartley,et al.  Minimal Solutions for Panoramic Stitching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.