Virtual worlds using computer vision

Virtual Reality has traditionally relied on hand-created synthetic virtual worlds as approximations of real world spaces. Creation of such virtual worlds is very labour intensive. Computer vision has recently contributed greatly to the creation of the visual/graphical aspect of the virtual worlds. These techniques are classified under image-based-as opposed to geometry-based-rendering in computer graphics. Image based rendering (IBR) aims to recreate a visual world given a few real views of it. We survey some of the important image-based rendering techniques in this paper, analyzing their assumptions and limitations. We then discuss Virtualized Reality, our contribution to the creation of virtual worlds from dynamic events using a stereo technique that gives dense depth maps on all-around views of the event. The virtualized world can be represented as multiple view-dependent models or as a single view-independent model. It can then be synthesized visually given the position and properties of any virtual camera. We present a few results from 3D Dome, our virtualizing facility.

[1]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[2]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[3]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[4]  Takeo Kanade,et al.  Virtualized reality: concepts and early results , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[5]  Richard Szeliski,et al.  3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[7]  Ramesh C. Jain,et al.  Multiple perspective interactive video , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.

[8]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Takeo Kanade,et al.  A stereo machine for video-rate dense depth mapping and its new applications , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[11]  Takeo Kanade,et al.  Recovery of dynamic scene structure from multiple image sequences , 1996, 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems (Cat. No.96TH8242).

[12]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  E LorensenWilliam,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987 .

[14]  Jules Bloomenthal,et al.  Polygonization of implicit surfaces , 1988, Comput. Aided Geom. Des..

[15]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[17]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[18]  Jules Bloomenthal,et al.  An Implicit Surface Polygonizer , 1994, Graphics Gems.

[19]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[20]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[21]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[22]  Steven M. Seitz,et al.  Toward image-based scene representation using view morphing , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[23]  Amnon Shashua,et al.  Algebraic Functions For Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[25]  Hideyuki Tamura,et al.  Viewpoint-dependent stereoscopic display using interpolation of multiviewpoint images , 1995, Electronic Imaging.