Historical Perspectives on 4D Virtualized Reality

Recording dynamic events, such as a sports event, a ballet performance, or a lecture, digitally for experiencing in a spatiotemporally distant setting requires 4D capture: three dimensions for their geometry/ appearance over the fourth dimension of time. Cameras are suitable for this task as they are nonintrusive, universal, and inexpensive. Computer Vision techniques have advanced sufficiently to make the 4D capture possible. In this paper, we present a historical perspective on the Virtualized RealityTM system developed since early 90s to early 2000 at CMU for the 4D capture of dynamic events.

[1]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[2]  Takeo Kanade,et al.  Virtual Space Teleconferencing Using a Sea of Cameras , 1994 .

[3]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[4]  Takeo Kanade,et al.  Virtualized reality: constructing time-varying virtual worlds from real world events , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[5]  Takeo Kanade,et al.  Virtualized reality: concepts and early results , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[6]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[7]  Takeo Kanade,et al.  CMU Video-Rate Stereo Machine , 1995 .

[8]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[9]  Refractor.  Vision , 2000, The Lancet.

[10]  P. J. Narayanan,et al.  Compression of multiple depth maps for IBR , 2005, The Visual Computer.

[11]  Saied Moezzi,et al.  Virtual View Generation for 3D Digital Video , 1997, IEEE Multim..

[12]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[13]  Takeo Kanade.  Hidden Valley , PA Very Fast 3-D Sensing Hardware , 1993 .

[14]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[15]  Takeo Kanade,et al.  The 3D Room: Digitizing Time-Varying 3D Events by Synchronized Multiple Video Streams , 1998 .

[16]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[18]  Takeo Kanade,et al.  Recovery of dynamic scene structure from multiple image sequences , 1996, 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems (Cat. No.96TH8242).

[19]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[20]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[21]  Takeo Kanade,et al.  Video-Rate Z Keying: A New Method for Merging Images , 1995 .

[22]  P J Narayanan,et al.  Virtual ized Reality : Concepts and Early Results , 1995 .

[23]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[24]  Shing-Chow Chan,et al.  An object-based compression system for a class of dynamic image-based representations , 2005, Visual Communications and Image Processing.

[25]  Takeo Kanade,et al.  Virtual ized reality: constructing time-varying virtual worlds from real world events , 1997 .

[26]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[29]  Takeo Kanade,et al.  Appearance-based virtual view generation of temporally-varying events from multi-camera images in the 3D room , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[30]  Hideo Saito,et al.  Modeling, Combining, and Rendering Dynamic Real-World Events From Image Sequences , 1998 .

[31]  P. J. Narayanan,et al.  Depth Images: Representations and Real-Time Rendering , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[32]  Takeo Kanade,et al.  Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[33]  Markus H. Gross,et al.  Scalable 3D video of dynamic scenes , 2005, The Visual Computer.

[34]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[35]  Marc Rioux,et al.  Active Optical 3D Imaging for Heritage Applications , 2002, IEEE Computer Graphics and Applications.