Using Line and Ellipse Features for Rectification of Broadcast Hockey Video

To use hockey broadcast videos for automatic game analysis, we need to compensate for camera viewpoint and motion. This can be done by using features on the rink to estimate the homography between the observed rink and a geometric model of the rink, as specified in the appropriate rule book (top down view of the rink). However, player occlusion, wide range of camera motion, and frames with few reliable key-points all pose significant challenges for robustness and accuracy of the solution. In this work, we describe a new method to use line and ellipse features along with key-point based matches to estimate the homography. We combine domain knowledge (i.e., rink geometry) with an appearance model of the rink to detect these features accurately. This over determines the homography estimation to make the system more robust. We show this approach is applicable to real world data and demonstrate the ability to track long sequences on the order of 1,000 frames.

[1]  Robert J. Woodham,et al.  Video analysis of hockey play in selected game situations , 2009, Image Vis. Comput..

[2]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Robert A. McLaughlin,et al.  Randomized Hough Transform: Improved ellipse detection with comparison , 1998, Pattern Recognit. Lett..

[4]  Alan Fern,et al.  Improved Video Registration using Non-Distinctive Local Image Features , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Hui Zeng,et al.  A new normalized method on line-based homography estimation , 2008, Pattern Recognit. Lett..

[8]  Robert J. Woodham,et al.  Combining Line and Point Correspondences for Homography Estimation , 2008, ISVC.

[9]  Justus H. Piater,et al.  Robust incremental rectification of sports video sequences , 2004, BMVC.

[10]  James J. Little,et al.  AUTOMATIC RECTIFICATION OF LONG IMAGE SEQUENCES , 2003 .

[11]  H. Katzgraber.  Introduction to Monte Carlo Methods , 2009, 0905.1629.

[12]  Yeung Sam Hung,et al.  A Hierarchical Approach for Fast and Robust Ellipse Extraction , 2007, 2007 IEEE International Conference on Image Processing.

[13]  Akihiro Sugimoto.  A Linear Algorithm for Computing the Homography from Conics in Correspondence , 2004, Journal of Mathematical Imaging and Vision.

[14]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[15]  Andrew W. Fitzgibbon,et al.  Direct least squares fitting of ellipses , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[16]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Justus H. Piater,et al.  On-Line Rectification of Sport Sequences with Moving Cameras , 2007, MICAI.

[18]  Christos Conomis.  Conics-Based Homography Estimation from Invariant Points and Pole-Polar Relationships , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[19]  Justus H. Piater,et al.  Fast 2D model-to-image registration using vanishing points for sports video analysis , 2005, IEEE International Conference on Image Processing 2005.

[20]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[22]  Wolfgang Effelsberg,et al.  Robust camera calibration for sport videos using court models , 2003, IS&T/SPIE Electronic Imaging.

[23]  Lu Wang,et al.  Wide-baseline image matching using Line Signatures , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[24]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[25]  Josef Kittler,et al.  Detecting partially occluded ellipses using the Hough transform , 1989, Image Vis. Comput..

[26]  Alexander Behrens,et al.  Analysis of Feature Point Distributions for Fast Image Mosaicking Algorithms , 2010 .

[27]  Wenchao Cai,et al.  A fast contour-based approach to circle and ellipse detection , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[28]  PAUL D. SAMPSON,et al.  Fitting conic sections to "very scattered" data: An iterative refinement of the bookstein algorithm , 1982, Comput. Graph. Image Process..

[29]  Erkki Oja,et al.  A new curve detection method: Randomized Hough transform (RHT) , 1990, Pattern Recognit. Lett..

[30]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Changhai Xu,et al.  3D pose estimation for planes , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[32]  C. V. Jawahar,et al.  Geometric Structure Computation from Conics , 2004, ICVGIP.

[33]  Irfan A. Essa,et al.  Motion fields to predict play evolution in dynamic sport scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Juho Kannala,et al.  Algorithms for Computing a Planar Homography from Conics in Correspondence , 2006, BMVC.

[35]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[36]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[37]  W. Gander,et al.  Least-squares fitting of circles and ellipses , 1994 .

[38]  Luc Van Gool,et al.  Wide-baseline stereo matching with line segments , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[39]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[40]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[41]  Zhanyi Hu,et al.  Line matching leveraged by point correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Zhi-Qiang Liu,et al.  A robust, real-time ellipse detector , 2005, Pattern Recognit..

[43]  Wolfgang Förstner,et al.  Detecting interpretable and accurate scale-invariant keypoints , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[44]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..