Synthesizing Free-Viewpoing Images from Multiple View Videos in Soccer StadiumADIUM

We propose a new method for synthesizing free-viewpoint images from multiple view videos in soccer stadium. The previous method can synthesize virtual viewpoint images at only intermediate viewpoint between real cameras. The previous method estimates only pixel-wise correspondences between input images for synthesizing virtual viewpoint images without a 3D reconstruction. In this paper, we propose a new method that can synthesize virtual viewpoint images at free-viewpoints which are not limited at intermediate viewpoint by reconstructing simple 3D models. To reconstruct the object soccer scene, we perform camera calibration without marker objects because we can save labor to put such objects on the soccer ground. For reconstructing of dynamic regions, we employ a billboard representation of dynamic regions. The position of the billboard is determined by tracking the players. The billboard representation can realize a simple 3D reconstruction and transform textures into the appropriate appearance of the aspect of dynamic regions at virtual viewpoint. We also introduce the choice of optimal cameras for rendering on the billboard. We choose the optimal camera that has the most similar texture at virtual viewpoint. By applying our method to a soccer scene, we can synthesize free-viewpoint images by employing only pixel values of real images

[1]  Saied Moezzi,et al.  Virtual View Generation for 3D Digital Video , 1997, IEEE Multim..

[2]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[3]  Adrian Hilton,et al.  Model-based multiple view reconstruction of people , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Takeo Kanade,et al.  Recording of multiple videos in a large-scale space for large-scale virtualized reality , 2001 .

[5]  Hideo Saito,et al.  Video synthesis at tennis player viewpoint from multiple view videos , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[6]  Takeo Kanade,et al.  Virtualized reality: concepts and early results , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[7]  Hideo Saito,et al.  Intermediate view generation of soccer scene from multiple videos , 2001, Object recognition supported by user interaction for service robots.

[8]  Oliver Grau,et al.  A combined studio production system for 3-D capturing of live action and immersive actor feedback , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Hideo Saito,et al.  Arbitrary-view image generation from multiple silhouette images in projective grid space , 2000, IS&T/SPIE Electronic Imaging.

[10]  Yuichi Ohta,et al.  Free viewpoint browsing of live soccer games , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[11]  Kazuo Fukui,et al.  Robust tracking of soccer players based on data fusion , 2002, Object recognition supported by user interaction for service robots.