3-D Reconstruction of Urban Scenes from Sequences of Images

In this paper, we address the problem of the recovery of the Euclidean geometry of a scene from a sequence of images without any prior knowledge either about the parameters of the cameras, or about the motion of the cam-era(s). We do not require any knowledge of the absolute coordinates of some control points in the scene to achieve this goal. Using various computer vision tools, we establish correspondences between images and recover the epipolar geometry of the set of images, from which we show how to compute the complete set of perspective projection matrices for each camera position. These being known, we proceed to reconstruct the scene. This reconstruction is defined up to an unknown projective transformation (i.e. is parameterized with 15 arbitrary parameters). Next we show how to go from this reconstruction to a more constrained class of reconstructions, defined up to an unknown affine transformation (i.e. parameterized with 12 arbitrary parameters) by exploiting known geometric relations between features in the scene such as parallelism. Finally, we show how to go from this reconstruction to another class, defined up to an unknown similitude (i.e. parameterized with 7 arbitrary parameters). This means that in an Euclidean frame attached to the scene or to one of the cameras, the reconstruction depends only upon one parameter, the global scale. This parameter is easily fixed as soon as one absolute length measurement is known. We see this vision system as a building block, a vision server, of a CAD system that is used by a human to model a scene for such applications as simulation, virtual or augmented reality. We believe that such a system can save a lot of tedious work to the human observer as well as play a leading role in keeping the geometric data base accurate and coherent.

[1]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[2]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[3]  Soren W. Henriksen,et al.  Manual of photogrammetry , 1980 .

[4]  Azriel Rosenfeld,et al.  Gray-level corner detection , 1982, Pattern Recognit. Lett..

[5]  J. Alison Noble,et al.  Finding Corners , 1988, Alvey Vision Conference.

[6]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[7]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[8]  Antonio Guiducci,et al.  Corner characterization by differential geometry techniques , 1988, Pattern Recognit. Lett..

[9]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[10]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[12]  Søren I. Olsen,et al.  Epipolar Line Estimation , 1992, ECCV.

[13]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[14]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[15]  Giulio Sandini,et al.  2nd European conference on computer vision , 1992, Image Vis. Comput..

[16]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[18]  Quang-Tuan Luong.  Matrice Fondamentale et Calibration Visuelle sur l''''Environnement - Vers une plus grande autonomie , 1992 .

[19]  Eamon B. Barrett,et al.  Some invariant linear methods in photogrammetry and model-matching , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Andrew Zisserman,et al.  Applications of Invariance in Computer Vision , 1993, Lecture Notes in Computer Science.

[21]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[22]  O. Faugeras,et al.  On determining the fundamental matrix : analysis of different methods and experimental results , 1993 .

[23]  Rachid Deriche,et al.  Recovering and characterizing image features using an efficient model based approach , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[25]  Olivier D. Faugeras,et al.  A Stability Analysis of the Fundamental Matrix , 1994, ECCV.

[26]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[27]  Thierry Viéville,et al.  Canonic Representations for the Geometries of Multiple Projective Views , 1994, ECCV.

[28]  Olivier D. Faugeras,et al.  A comparison of projective reconstruction methods for pairs of views , 1995, Proceedings of IEEE International Conference on Computer Vision.

[29]  Roger MohrLifia.  Epipole and Fundamental Matrix Estimation Using the Virtual Parallax Property , 1995 .

[30]  Roger Mohr,et al.  Epipole and fundamental matrix estimation using virtual parallax , 1995, Proceedings of IEEE International Conference on Computer Vision.

[31]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[32]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[33]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[34]  Olivier D. Faugeras,et al.  A Comparison of Projective Reconstruction Methods for Pairs of Views , 1997, Comput. Vis. Image Underst..

[35]  Olivier D. Faugeras,et al.  Characterizing the Uncertainty of the Fundamental Matrix , 1997, Comput. Vis. Image Underst..

[36]  Olivier D. Faugeras,et al.  Camera Calibration, Scene Motion and Structure recovery from point correspondences and fundamental m , 1997 .