Dual-Mode Deformable Models for Free-Viewpoint Video of Sports Events

Generating free-viewpoint video in outdoor sports environments is currently an unsolved problem due to difficulties in obtaining accurate background segmentation and camera calibration. This paper introduces a technique for the reconstruction of a scene in the presence of these errors. We tackle the issues of reconstruction completeness, and accuracy of surface shape and appearance. We introduce the concept of the conservative visual hull as a technique to improve reconstruction completeness. We then present a view-dependent surface optimisation technique using deformable models to improve surface shape and appearance. We contribute a novel dual-mode snake algorithm that is robust to noise and demonstrates reduced dependence on parameterisation by separating the search of the solution space from the data fitting. We conclude by presenting results of this technique along with a quantitative evaluation against other reconstruction techniques using a leave-one- out data set.

[1]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[2]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[3]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[4]  Ian D. Reid,et al.  A Multiple View Layered Representation for Dynamic Novel View Synthesis , 2003, BMVC.

[5]  Jian Sun,et al.  Poisson matting , 2004, ACM Trans. Graph..

[6]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[7]  I. Kitahara,et al.  Live mixed-reality 3D video in soccer stadium , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[8]  Hideo Saito,et al.  Synthesizing Free-Viewpoing Images from Multiple View Videos in Soccer StadiumADIUM , 2006, International Conference on Computer Graphics, Imaging and Visualisation (CGIV'06).

[9]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[10]  Hideo Saito,et al.  Arbitrary viewpoint observation for soccer match video , 2004 .

[11]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[12]  Stanley Osher,et al.  Level Set Methods , 2003 .

[13]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Adrian Hilton,et al.  Virtual view synthesis of people from multiple view video sequences , 2005, Graph. Model..

[16]  Demetri Terzopoulos,et al.  Symmetry-seeking models and 3D object reconstruction , 1988, International Journal of Computer Vision.

[17]  Alper Yilmaz,et al.  Level Set Methods , 2007, Wiley Encyclopedia of Computer Science and Engineering.

[18]  Adrian Hilton,et al.  A Comparative Study of Free-Viewpoint Video Techniques For sports events , 2006 .

[19]  G. A. Thomas,et al.  Real-Time Camera Pose Estimation for Augmenting Sports Scenes , 2006 .

[20]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..