3D Reconstruction from Multiple Images: Part 1 - Principles

This issue discusses methods to extract three-dimensional (3D) models from plain images. In particular, the 3D information is obtained from images for which the camera parameters are unknown. The principles underlying such uncalibrated structure-from-motion methods are outlined. First, a short review of 3D acquisition technologies puts such methods in a wider context and highlights their important advantages. Then, the actual theory behind this line of research is given. The authors have tried to keep the text maximally self-contained, therefore also avoiding to rely on an extensive knowledge of the projective concepts that usually appear in texts about self-calibration 3D methods. Rather, mathematical explanations that are more amenable to intuition are given. The explanation of the theory includes the stratification of reconstructions obtained from image pairs as well as metric reconstruction on the basis of more than two images combined with some additional knowledge about the cameras used. Readers who want to obtain more practical information about how to implement such uncalibrated structure-from-motion pipelines may be interested in two more Foundations and Trends issues written by the same authors. Together with this issue they can be read as a single tutorial on the subject.

[1]  Thierry Viéville,et al.  Canonical Representations for the Geometries of Multiple Projective Views , 1996, Comput. Vis. Image Underst..

[2]  Gideon P. Stein,et al.  Lens distortion calibration using point correspondences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Thierry Viéville,et al.  Canonic Representations for the Geometries of Multiple Projective Views , 1994, ECCV.

[4]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.

[6]  Bill Triggs,et al.  Matching constraints and the joint image , 1995, Proceedings of IEEE International Conference on Computer Vision.

[7]  Hanumant Singh,et al.  Relative Pose Estimation for Instrumented, Calibrated Imaging Platforms , 2003, DICTA.

[8]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[10]  Jianhua Wang,et al.  A New Calibration Model and Method of Camera Lens Distortion , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[12]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[13]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  Luc Van Gool,et al.  Affine Reconstruction from Perspective Image Pairs Obtained by a Translating Camera , 1993, Applications of Invariance in Computer Vision.

[16]  Donald P. Greenberg,et al.  Non-linear approximation of reflectance functions , 1997, SIGGRAPH.

[17]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.

[19]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[21]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Richard I. Hartley.  Self-Calibration from Multiple Views with a Rotating Camera , 1994, ECCV.

[23]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  H. C. Longuet-Higgins.  The reconstruction of a plane surface from two perspective projections , 1986, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[25]  L. Gool,et al.  Affine reconstruction from perspective image pairs , 1993 .

[26]  Olivier D. Faugeras,et al.  The Critical Sets of Lines for Camera Displacement Estimation: A Mixed Euclidean-Projective and Constructive Approach , 2004, International Journal of Computer Vision.

[27]  Roger Tsai,et al.  Synopsis of recent progress on camera calibration for 3D machine vision , 1989 .

[28]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[29]  Olivier Faugeras,et al.  Automatic calibration and removal of distortion from scenes of structured environments , 1995, Optics & Photonics.

[30]  Richard I. Hartley,et al.  A linear method for reconstruction from lines and points , 1995, Proceedings of IEEE International Conference on Computer Vision.

[31]  Peter F. Sturm,et al.  Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length , 1999, Image Vis. Comput..

[32]  J. Philip.  A non-iterative algorithm for determining all essential matrices corresponding to five point pairs , 1996 .

[33]  Sing Bing Kang,et al.  Parameter-Free Radial Distortion Correction with Center of Distortion Estimation , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[35]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[36]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[37]  Steven K. Feiner,et al.  Introduction to Computer Graphics , 1993 .

[38]  Luc Van Gool,et al.  Affine Reconstruction from Perspective Image Pairs with a Relative Object-Camera Translation in Between , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Songde Ma,et al.  Implicit and Explicit Camera Calibration: Theory and Experiments , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Thomas S. Huang,et al.  Theory of Reconstruction from Image Motion , 1992 .

[42]  Peter F. Sturm,et al.  Calibration of Cameras with Radially Symmetric Distortion , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[44]  David Nister,et al.  Automatic Dense Reconstruction from Uncalibrated Video Sequences , 2001 .

[45]  Peter Sturm.  Critical Motion Sequences and Conjugacy of Ambiguous Euclidean Reconstructions , 1997 .

[46]  B. Triggs.  The Geometry of Projective Reconstruction I: Matching Constraints and the Joint Image , 1995 .

[47]  Olivier D. Faugeras,et al.  Automatic calibration and removal of distortion from scenes of structured environments , 1995, Optics & Photonics.

[48]  Amnon Shashua,et al.  Ambiguity in reconstruction from images of six points , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[49]  T. Clarke,et al.  The Development of Camera Calibration Methods and Models , 1998 .

[50]  Stephen J. Maybank,et al.  The critical line congruence for reconstruction from three images , 1995, Applicable Algebra in Engineering, Communication and Computing.

[51]  Richard I. Hartley,et al.  In defence of the 8-point algorithm , 1995, Proceedings of IEEE International Conference on Computer Vision.

[52]  Olivier D. Faugeras,et al.  The fundamental matrix: Theory, algorithms, and stability analysis , 2004, International Journal of Computer Vision.

[53]  Bill Triggs,et al.  Critical Motions for Auto-Calibration When Some Intrinsic Parameters Can Vary , 2000, Journal of Mathematical Imaging and Vision.

[54]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[55]  Olivier D. Faugeras,et al.  From projective to Euclidean reconstruction , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  Zhengyou Zhang,et al.  On the epipolar geometry between two images with lens distortion , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[57]  Gene H. Golub,et al.  Matrix computations , 1983 .

[58]  Olivier D. Faugeras,et al.  The critical sets of lines for camera displacement estimation: A mixed Euclidean-projective and constructive approach , 1993, 1993 (4th) International Conference on Computer Vision.

[59]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[60]  R. Hartley.  Triangulation, Computer Vision and Image Understanding , 1997 .

[61]  Olivier D. Faugeras,et al.  Oriented Projective Geometry for Computer Vision , 1996, ECCV.

[62]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.