A Survey of Methods for Volumetric Scene Reconstruction from Photographs

Scene reconstruction, the task of generating a 3D model of a scene given multiple 2D photographs taken of the scene, is an old and difficult problem in computer vision. Since its introduction, scene reconstruction has found application in many fields, including robotics, virtual reality, and entertainment. Volumetric models are a natural choice for scene reconstruction. Three broad classes of volumetric reconstruction techniques have been developed based on geometric intersections, color consistency, and pair-wise matching. Some of these techniques have spawned a number of variations and undergone considerable refinement. This paper is a survey of techniques for volumetric scene reconstruction.

[1]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[2]  Qian Chen,et al.  A volumetric stereo matching method: application to image-based modeling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Narendra Ahuja,et al.  Octree Generation from Object Silhouettes in Perspective Views , 1990, Comput. Vis. Graph. Image Process..

[4]  Ramesh C. Jain,et al.  Reality modeling and visualization from multiple video sequences , 1996, IEEE Computer Graphics and Applications.

[5]  Saied Moezzi,et al.  Virtual View Generation for 3D Digital Video , 1997, IEEE Multim..

[6]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[8]  Thomas Fromherz,et al.  Shape from Multiple Cues: Integrating Local Brightness Information , 1995 .

[9]  Pere Brunet,et al.  3D reconstruction with projective octrees and epipolar geometry , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Alan L. Yuille,et al.  Local, global, and multilevel stereo matching , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Peter Eisert,et al.  Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[12]  Arie Kaufman,et al.  Volume Visualization (Tutorial) , 1991 .

[13]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[14]  Kiriakos N. Kutulakos,et al.  What Do N Photographs Tell Us about 3D Shape , 1998 .

[15]  Peter Eisert,et al.  3-D object reconstruction using spatially extended voxels and multi-hypothesis voxel coloring , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[16]  Jake K. Aggarwal,et al.  Volume/surface octrees for the representation of three-dimensional objects , 1986, Comput. Vis. Graph. Image Process..

[17]  C. Dyer.  Volumetric Scene Reconstruction from Multiple Views , 2001 .

[18]  J. A. Sethian,et al.  Fast Marching Methods , 1999, SIAM Rev..

[19]  Nelson L. Max,et al.  Hierarchical Rendering of Trees from Precomputed Multi-Layer Z-Buffers , 1996, Rendering Techniques.

[20]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[21]  Takeo Kanade,et al.  3D voxel construction based on epipolar geometry , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[22]  Narendra Ahuja,et al.  EFFICIENT OCTREE GENERATION FROM SILHOUETTES. , 1986 .

[23]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[24]  Lina Massone,et al.  Shape From Occluding Contours , 1985, Other Conferences.

[25]  James F. Greenleaf,et al.  Computer Generated Three-Dimensional Oscilloscopic Images and Associated Techniques for Display and Study of the Spatial Distribution of Pulmonary Blood Flow , 1970 .

[26]  Peter Eisert,et al.  3-D reconstruction of real-world objects using extended voxels , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[27]  Steven M. Seitz,et al.  Complete scene structure from four point correspondences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[28]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[29]  Takeo Kanade,et al.  Shape and motion carving in 6D , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[30]  Thomas Malzbender,et al.  Improved Voxel Coloring Via Volumetric Optimization , 2001 .

[31]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[32]  Takeo Kanade,et al.  Shape reconstruction in projective grid space from large number of images , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[33]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[34]  Ernest W. Kent,et al.  Representing Workspace and Model Knowledge for a Robot with Mobile Sensors | NIST , 1984 .

[35]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[36]  Thomas Fromherz,et al.  Shape from contours as initial step in shape from multiple cues , 1994, Other Conferences.

[37]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[38]  Arie E. Kaufman.  Volume visualization , 1996, CSUR.

[39]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[40]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[41]  Larry S. Davis.  Foundations of Image Understanding , 2001 .

[42]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[43]  Thomas Malzbender,et al.  Volumetric Warping for Voxel Coloring on an Infinite Domain , 2000, SMILE.

[44]  Donald P. Greenberg,et al.  Improved Computational Methods for Ray Tracing , 1984, TOGS.

[45]  Kiriakos N. Kutulakos.  Approximate N-View Stereo , 2000, ECCV.

[46]  J. Sethian,et al.  FRONTS PROPAGATING WITH CURVATURE DEPENDENT SPEED: ALGORITHMS BASED ON HAMILTON-JACOB1 FORMULATIONS , 2003 .

[47]  Dimitris N. Metaxas,et al.  Dynamic 3D models with local and global deformations: deformable superquadrics , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[48]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[49]  Michael Potmesil.  Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[50]  Michael Werman,et al.  Trilinearity of three perspective views and its associated tensor , 1995, Proceedings of IEEE International Conference on Computer Vision.

[51]  Paul A. Viola,et al.  Roxels: responsibility weighted 3D volume reconstruction , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[52]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[53]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..