A probabilistic framework for space carving

This paper introduces a new probabilistic framework for Space Carving. In this framework each voxel is assigned a probability, which is computed by comparing the likelihoods for the voxel existing and not existing. This new framework avoids many of the difficulties associated with the original Space Carving algorithm. Specifically, it does not need a global threshold parameter, and it guarantees that no holes will be carved in the model. This paper also proposes that a voxel-based thick texture is a realistic and efficient representation for scenes which contain dominant planes. The algorithm is tested using both real and synthetic data, and both qualitative and quantitative results are presented.

[1]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[2]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[3]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[4]  Robert F. Sproull,et al.  Principles in interactive computer graphics , 1973 .

[5]  Azriel Rosenfeld,et al.  Scene Labeling by Relaxation Operations , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[7]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[8]  B. Julesz,et al.  A disparity gradient limit for binocular fusion. , 1980, Science.

[9]  Thomas O. Binford,et al.  Depth from Edge and Intensity Based Stereo , 1981, IJCAI.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  John E. W. Mayhew,et al.  Psychophysical and Computational Studies Towards a Theory of Human Stereopsis , 1981, Artif. Intell..

[12]  W E Grimson,et al.  A computer implementation of a theory of human stereo vision. , 1981, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[14]  Ian J. Hayes.  Computer architecture: the hardware - software interface , 1983 .

[15]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[16]  M Rioux,et al.  Laser range finder based on synchronized scanners. , 1984, Applied optics.

[17]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[18]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[21]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[22]  Hiroshi Noborio,et al.  Construction of the Octree Approximating a Three-Dimensional Object by Using Multiple Views , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Roger Y. Tsai,et al.  Techniques for Calibration of the Scale Factor and Image Center for High Accuracy 3-D Machine Vision Metrology , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[25]  Narendra Ahuja,et al.  Generating Octrees from Object Silhouettes in Orthographic Views , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Narendra Ahuja,et al.  Octree Generation from Object Silhouettes in Perspective Views , 1990, Comput. Vis. Graph. Image Process..

[27]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[28]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  David Mumford,et al.  A Bayesian treatment of the stereo correspondence problem using half-occluded regions , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Quang-Tuan Luong.  Matrice Fondamentale et Calibration Visuelle sur l''''Environnement - Vers une plus grande autonomie , 1992 .

[31]  A D Marshall,et al.  Geometric Modelling for Computer Vision , 1992 .

[32]  Peter N. Belhumeur,et al.  A binocular stereo algorithm for reconstructing sloping, creased, and broken surfaces in the presence of half-occlusion , 1993, 1993 (4th) International Conference on Computer Vision.

[33]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[34]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[35]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[36]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Long Quan,et al.  Invariants of 6 Points from 3 Uncalibrated Images , 1994, ECCV.

[38]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[39]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Aaron F. Bobick,et al.  Disparity-Space Images and Large Occlusion Stereo , 1994, ECCV.

[41]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[42]  Reinhard Koch,et al.  3-D surface reconstruction from stereoscopic image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[43]  Steven M. Seitz,et al.  Physically-valid view synthesis by image interpolation , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[44]  Michael Werman,et al.  Trilinearity of three perspective views and its associated tensor , 1995, Proceedings of IEEE International Conference on Computer Vision.

[45]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[46]  Richard I. Hartley,et al.  A linear method for reconstruction from lines and points , 1995, Proceedings of IEEE International Conference on Computer Vision.

[47]  Aldo Laurentini,et al.  How Far 3D Shapes Can Be Understood from 2D Silhouettes , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[49]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[50]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[51]  Jitendra Malik,et al.  On Binocularly Viewed Occlusion Junctions , 1996, ECCV.

[52]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[54]  Steven M. Seitz,et al.  Image-based transformation of viewpoint and scene appearance , 1997 .

[55]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[57]  Cordelia Schmid,et al.  Automatic line matching across views , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[58]  Avideh Zakhor,et al.  View generation for three-dimensional scenes from video sequences , 1997, IEEE Trans. Image Process..

[59]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[60]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[61]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[62]  Reinhard Koch,et al.  Metric 3D Surface Reconstruction from Uncalibrated Image Sequences , 1998, SMILE.

[63]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[64]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.

[65]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[66]  Olivier D. Faugeras,et al.  A nonlinear method for estimating the projective geometry of 3 views , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[67]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[68]  Anders Heyden,et al.  A Common Framework for Multiple View Tensors , 1998, ECCV.

[69]  Andrew Zisserman,et al.  Resolving ambiguities in auto–calibration , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[70]  Sean Hayes,et al.  View synthesis by trinocular edge matching and transfer , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[71]  Richard I. Hartley,et al.  Minimizing algebraic error in geometric estimation problems , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[72]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[73]  Roberto Cipolla,et al.  3D models of architectural scenes from uncalibrated images and vanishing points , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[74]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[75]  Takeo Kanade,et al.  3D voxel construction based on epipolar geometry , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[76]  Richard Szeliski,et al.  An integrated Bayesian approach to layer extraction from image sequences , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[77]  Jeremy S. De Bonet,et al.  Poxels: Probabilistic Voxelized Volume Reconstruction , 1999 .

[78]  Roberto Cipolla,et al.  Calibration of image sequences for model visualisation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[79]  Richard Szeliski.  Stereo Algorithms and Representations for Image-based Rendering , 1999, BMVC.

[80]  Paulo R. S. Mendonça,et al.  A simple technique for self-calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[81]  Qian Chen,et al.  A volumetric stereo matching method: application to image-based modeling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[82]  Takeo Kanade,et al.  Shape reconstruction in projective grid space from large number of images , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[83]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[84]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[85]  Kiriakos N. Kutulakos.  Approximate N-View Stereo , 2000, ECCV.

[86]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[87]  Roberto Cipolla,et al.  A Statistical Consistency Check for the Space Carving Algorithm , 2000, BMVC.

[88]  Andrew Zisserman,et al.  A Six Point Solution for Structure and Motion , 2000, ECCV.

[89]  P. Torr,et al.  Layer Extraction with a Bayesian Model of Shapes , 2000, ECCV.

[90]  Paul Smith,et al.  Motion Segmentation by Tracking Edge Information over Multiple Frames , 2000, ECCV.

[91]  Klaus Mueller,et al.  A practical evaluation of popular volume rendering algorithms , 2000, VVS '00.

[92]  R. Zabih,et al.  Exact voxel occupancy with graph cuts , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[93]  Andrew Zisserman,et al.  Surface Reconstruction from Multiple Views Using Apparent Contours and Surface Texture , 2000, Confluence of Computer Vision and Computer Graphics.

[94]  Thomas Malzbender,et al.  Volumetric Warping for Voxel Coloring on an Infinite Domain , 2000, SMILE.

[95]  C. Dyer.  Volumetric Scene Reconstruction from Multiple Views , 2001 .

[96]  Roberto Cipolla,et al.  Structure and motion from silhouettes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[97]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[98]  Thomas Malzbender,et al.  A Survey of Methods for Volumetric Scene Reconstruction from Photographs , 2001, VG.

[99]  Paulo R. S. Mendonça,et al.  Epipolar geometry from profiles under circular motion , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[100]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[101]  William H. Press,et al.  Numerical recipes in C , 2002 .