Dual Mode Deformable Models for Free-Viewpoint Video of Outdoor Sports Events

Generating free-viewpoint video in outdoor sports environments is currently an unsolved problem due to difficulties in obtaining accurate background segmentation and camera calibration. This paper introduces a technique for the reconstruction of a scene in the presence of these errors. We tackle the issues of reconstruction completeness, and accuracy of surface shape and appearance. We introduce the concept of the conservative visual hull as a technique to improve reconstruction completeness. We then present a view-dependent surface optimisation technique using deformable models to improve surface shape and appearance. We contribute a novel dual-mode snake algorithm that is robust to noise and demonstrates reduced dependence on parameterisation by separating the search of the solution space from the data fitting. We conclude by presenting results of this technique along with a quantitative evaluation against other reconstruction techniques using a leave-oneout data set.

[1]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[2]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Hideo Saito,et al.  Arbitrary viewpoint observation for soccer match video , 2004 .

[4]  Hideo Saito,et al.  Synthesizing Free-Viewpoing Images from Multiple View Videos in Soccer StadiumADIUM , 2006, International Conference on Computer Graphics, Imaging and Visualisation (CGIV'06).

[5]  I. Kitahara,et al.  Live mixed-reality 3D video in soccer stadium , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[6]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[7]  Adrian Hilton,et al.  A Comparative Study of Free-Viewpoint Video Techniques For sports events , 2006 .

[8]  G. A. Thomas.  Real-Time Camera Pose Estimation for Augmenting Sports Scenes , 2006 .

[9]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[10]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Alper Yilmaz,et al.  Level Set Methods , 2007, Wiley Encyclopedia of Computer Science and Engineering.

[12]  Adrian Hilton,et al.  Virtual view synthesis of people from multiple view video sequences , 2005, Graph. Model..

[13]  Demetri Terzopoulos,et al.  Symmetry-seeking models and 3D object reconstruction , 1988, International Journal of Computer Vision.

[14]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[15]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[16]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[17]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[18]  Stanley Osher,et al.  Level Set Methods , 2003 .

[19]  Ian D. Reid,et al.  A Multiple View Layered Representation for Dynamic Novel View Synthesis , 2003, BMVC.

[20]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.