Live mixed-reality 3D video in soccer stadium

This paper proposes a method to realize a 3D video display system that can capture video from multiple cameras, reconstruct 3D models and transmit 3D video data in real time. We represent a target object with a simplified 3D model consisting of a single plane and a 2D texture extracted from multiple cameras. This 3D model is simple enough to be transmitted via a network. We have developed a prototype system that can capture multiple videos, reconstruct 3D models, transmit the models via a network, and display 3D video in real time. A 3D video of a typical soccer scene that includes a dozen players was processed at 26 frames per second.

[1]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[2]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[3]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[4]  Jean Ponce,et al.  Automatic Model Construction and Pose Estimation From Photographs Using Triangular Splines , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  Tomas Akenine-Möller,et al.  Real-time rendering , 1997 .

[7]  Michael Potmesil.  Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[8]  Hideo Saito,et al.  Large-scale Virtualized Reality , 2001 .

[9]  Ramesh C. Jain,et al.  An emerging medium: interactive three-dimensional digital video , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[10]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[11]  Anselmo Lastra,et al.  Creating Adaptive Views for Group Video Teleconferencing – An Image-Based Approach , 2002 .

[12]  Markus H. Gross,et al.  3D video recorder , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[13]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Takeo Kanade,et al.  Shape reconstruction in projective grid space from large number of images , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  L. Davis,et al.  M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene , 2003, International Journal of Computer Vision.

[16]  Tomas Akenine-Möller,et al.  Real-Time Rendering, Second Edition , 2002 .

[17]  Xiaojun Wu,et al.  Homography based parallel volume intersection: toward real-time volume reconstruction using active cameras , 2000, Proceedings Fifth IEEE International Workshop on Computer Architectures for Machine Perception.