Least-squares 3D reconstruction from one or more views and geometric clues

We present a method to reconstruct from one or more images a scene that is rich in planes, alignments, symmetries, orthogonalities, and other forms of geometrical regularity. Given image points of interest and some geometric information, the method recovers least-squares estimates of the 3D points, camera position(s), orientation(s), and eventually calibration(s). Our contributions lie (i) in a novel way of exploiting some types of symmetry and of geometric regularity, (ii) in treating indifferently one or more images, (iii) in a geometric test that indicates whether the input data uniquely defines a reconstruction, and (iv) a parameterization method for collections of 3D points subject to geometric constraints. Moreover, the reconstruction algorithm lends itself to sensitivity analysis. The method is benchmarked on synthetic data and its effectiveness is shown on real-world data.

[1]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Gunnar Sparr,et al.  Euclidean and Affine Structure/Motion for Uncalibrated Cameras from Affine Shape and Subsidiary Information , 1998, SMILE.

[3]  Subhajit Sanyal,et al.  Multilevel modelling and rendering of architectural scenes , 2003, Eurographics.

[4]  Ian D. Reid,et al.  Goal-directed Video Metrology , 1996, ECCV.

[5]  Daphna Weinshall,et al.  Using Bilateral Symmetry to Improve 3D Reconstruction from Image Sequences , 1997, Comput. Vis. Image Underst..

[6]  William H. Press,et al.  Numerical recipes , 1990 .

[7]  B. Caprile,et al.  Using vanishing points for camera calibration , 1990, International Journal of Computer Vision.

[8]  Bernard Mourrain,et al.  An Application of Automatic Theorem Proving in Computer Vision , 1998, Automated Deduction in Geometry.

[9]  Mei Han,et al.  Interactive construction of 3D models from panoramic mosaics , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[10]  E. Grossmann,et al.  Single and Multi-View Reconstruction of Structured Scenes , 2002 .

[11]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[12]  Robert B. Fisher,et al.  Improving architectural 3D reconstruction by plane and edge constraining , 2002, BMVC.

[13]  Adrien Bartoli,et al.  Constrained Structure and Motion From Multiple Uncalibrated Views of a Piecewise Planar Scene , 2003, International Journal of Computer Vision.

[14]  Antonio Criminisi,et al.  Accurate Visual Metrology from Single and Multiple Uncalibrated Images , 2001, Distinguished Dissertations.

[15]  José Santos-Victor,et al.  Dual Representations for Vision-Based 3D Reconstruction , 2000, BMVC.

[16]  G. Stewart.  Introduction to matrix computations , 1973 .

[17]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[18]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[19]  Roberto Cipolla,et al.  Automatic 3D Modelling of Architecture , 2000, BMVC.

[20]  Roberto Cipolla,et al.  An Interactive System for Constraint-Based Modelling , 2000, British Machine Vision Conference.

[21]  W. Rheinboldt.  On the computation of multi-dimensional solution manifolds of parametrized equations , 1988 .

[22]  D. Forsyth,et al.  Extracting Projective Information from Single Views of 3D Point Sets , 1993 .

[23]  Robert M. Haralick,et al.  Propagating covariance in computer vision , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[24]  Carsten Rother.  Linear multiview reconstruction of points, lines, planes and cameras using a reference plane , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[25]  Roberto Cipolla,et al.  3D Model Acquisition from Uncalibrated Images , 1998, MVA.

[26]  Peter F. Sturm,et al.  Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds , 2001, ICCV.

[27]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[28]  Zhanyi Hu,et al.  Single view metrology from scene constraints , 2005, Image Vis. Comput..

[29]  F. A. Heuvel,et al.  TRENDS IN CAD-BASED PHOTOGRAMMETRIC MEASUREMENT , 2000 .

[30]  Zhanyi Hu,et al.  Camera calibration and 3D reconstruction from a single view based on scene constraints , 2005, Image Vis. Comput..

[31]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  J. Gaspar,et al.  Interactive Reconstruction from an Omnidirectional Image , 2001 .

[33]  P. Sturm.  A method for 3D reconstruction of piecewise planar objects from single panoramic images , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[34]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[35]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[36]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[37]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[38]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[39]  Stephen J. Maybank,et al.  A Method for Interactive 3D Reconstruction of Piecewise Planar Objects from Single Images , 1999, BMVC.

[40]  Reinhard Koch,et al.  3D Structure from Multiple Images of Large-Scale Environments , 1998, Lecture Notes in Computer Science.

[41]  Didier Bondyfalat,et al.  Imposing Euclidean Constraints During Self-Calibration Processes , 1998, SMILE.

[42]  Camillo J. Taylor,et al.  Reconstruction of Linearly Parameterized Models from Single Images with a Camera of Unknown Focal Length , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Ricardo D. Fierro,et al.  The Total Least Squares Problem: Computational Aspects and Analysis (S. Van Huffel and J. Vandewalle) , 1993, SIAM Rev..

[44]  Mei Han,et al.  Interactive 3D Modeling from Multiple Images Using Scene Regularities , 1998, SMILE.

[45]  José Santos-Victor,et al.  Maximum Likelihood 3D Reconstruction from One or More Images under Geometric Constraints , 2002, BMVC.

[46]  Henrik I. Christensen,et al.  Multiple Plane Segmentation Using Optical Flow , 2002, BMVC.

[47]  Sylvain Petitjean,et al.  Algebraic Geometry and Computer Vision: Polynomial Systems, Real and Complex Roots , 1999, Journal of Mathematical Imaging and Vision.

[48]  Michel Dhome,et al.  Architectural Reconstruction with Multiple Views and Geometric Constraints , 2003, BMVC.

[49]  V. Mehrmann,et al.  Smooth factorizations of matrix valued functions and their derivatives , 1991 .

[50]  Allen Y. Yang,et al.  On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image , 2004, International Journal of Computer Vision.

[51]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[52]  Roger Mohr,et al.  Euclidean constraints for uncalibrated reconstruction , 1993, 1993 (4th) International Conference on Computer Vision.

[53]  Antonio Criminisi,et al.  Creating Architectural Models from Images , 1999, Comput. Graph. Forum.