Real-Time Video-Based Modeling and Rendering of 3D Scenes

Our work targets 3D scenes in motion. In this article, we propose a method for view-dependent layered representation of 3D dynamic scenes. Using densely arranged cameras, we've developed a system that can perform processing in real time from image pickup to interactive display, using video sequences instead of static images, at 10 frames per second. In our system, images on layers are view dependent, and we update both the shape and image of each layer in real time. This lets us use the dynamic layers as the coarse structure of the dynamic 3D scenes, which improves the quality of the synthesized images. In this sense, our prototype system may be one of the first full real-time image -based modelling and rendering systems. Our experimental results show that this method is useful for interactive 3D rendering of real scenes.

[1]  Masahiko Inami,et al.  Real-time rendering system of moving objects , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[2]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[3]  H Harashima,et al.  3-D computer graphics based on integral photography. , 2001, Optics express.

[4]  Takeshi Naemura,et al.  Multi-user immersive stereo , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[5]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[6]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[7]  Takeshi Naemura,et al.  Real-time video-based rendering for augmented spatial communication , 1998, Electronic Imaging.

[8]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[9]  Takeshi Naemura,et al.  Ray-based approach to integrated 3D visual communication , 2001, Optics East.

[10]  T. Naemura,et al.  Real-Time Video-Based Rendering for Augumented Spatial Communications , 1999 .

[11]  Takeshi Naemura,et al.  Pixel independent random access image sensor for real time image-based rendering system , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[12]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[13]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.