Canonical Representations for the Geometries of Multiple Projective Views

This work is in the context of motion and stereo analysis. It presents a new unified representation which will be useful when dealing with multiple views in the case of uncalibrated cameras. Several levels of information might be considered, depending on the availability of information. Among other things, an algebraic description of the epipolar geometry ofNviews is introduced, as well as a framework for camera self-calibration, calibration updating, and structure from motion in an image sequence taken by a camera which is zooming and moving at the same time. We show how a special decomposition of a set of two or three general projection matrices, calledcanonical, enables us to build geometric descriptions for a system of cameras which are invariant with respect to a given group of transformations. These representations are minimal and capture completely the properties of each level of description considered, Euclidean (in the context of calibration, and in the context of structure from motion, which we distinguish clearly), affine, and projective, that we also relate to each other. In the last case, a new decomposition of the well-knownfundamental matrixis obtained. Dependencies, which appear when three or more views are available, are studied in the context of the canonic decomposition, and new composition formulas are established. The theory is illustrated by tutorial examples with real images.

[1]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[2]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[3]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[4]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch, II: Singular value decomposition , 1982 .

[5]  Stephen T. Barnard,et al.  Interpreting Perspective Image , 1983, Artif. Intell..

[6]  Jake K. Aggarwal,et al.  Determining vanishing points from perspective images , 1984, Comput. Vis. Graph. Image Process..

[7]  G. Toscani.  Systèmes de calibration et perception du mouvement en vision artificielle , 1987 .

[8]  O. Faugeras,et al.  Motion from point matches: Multiplicity of solutions , 1989, [1989] Proceedings. Workshop on Visual Motion.

[9]  Olivier D. Faugeras,et al.  Some Properties of the E Matrix in Two-View Motion Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Roger Tsai,et al.  Synopsis of recent progress on camera calibration for 3D machine vision , 1989 .

[11]  Long Quan,et al.  Determining perspective structures using hierarchical Hough transform , 1989, Pattern Recognit. Lett..

[12]  S. Maybank.  Properties of essential matrices , 1990, Int. J. Imaging Syst. Technol..

[13]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[14]  David A. Forsyth,et al.  Projectively invariant representations using implicit algebraic curves , 1991, Image Vis. Comput..

[15]  Beatrice Brillault-O'Mahony,et al.  New method for vanishing point detection , 1991, CVGIP Image Underst..

[16]  Olivier Faugeras,et al.  3D Dynamic Scene Analysis , 1992 .

[17]  P. Beardsley,et al.  Affine and Projective Structure from Motion , 1992 .

[18]  Olivier Faugeras,et al.  Three D-Dynamic Scene Analysis: A Stereo Based Approach , 1992 .

[19]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[20]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[21]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[23]  Eamon B. Barrett,et al.  Some invariant linear methods in photogrammetry and model-matching , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Enrico Grosso,et al.  Relative positioning with uncalibrated cameras , 1992 .

[25]  Thierry Viéville,et al.  Computation of ego-motion and structure from visual and inertial sensors using the vertical cue , 1993, 1993 (4th) International Conference on Computer Vision.

[26]  Phil L. Palmer,et al.  An Optimised Vanishing Point Detector , 1993, BMVC.

[27]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[28]  Long Quan.  Affine Stereo Calibration for Relative Affine Shape Reconstruction , 1993, BMVC.

[29]  Roger Mohr,et al.  Euclidean constraints for uncalibrated reconstruction , 1993, 1993 (4th) International Conference on Computer Vision.

[30]  O. Faugeras,et al.  On determining the fundamental matrix : analysis of different methods and experimental results , 1993 .

[31]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[32]  L. Dron.  Dynamic camera self-calibration from controlled motion sequences , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Michael Brady,et al.  Self-calibration of the intrinsic parameters of cameras for active vision systems , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Amnon Shashua,et al.  Projective depth: A geometric invariant for 3D reconstruction from two perspective/orthographic views and for visual recognition , 1993, 1993 (4th) International Conference on Computer Vision.

[35]  Rajiv Gupta,et al.  Computing matched-epipolar projections , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[36]  L. Gool,et al.  Affine reconstruction from perspective image pairs , 1993 .

[37]  Luc Van Gool,et al.  Affine Reconstruction from Perspective Image Pairs Obtained by a Translating Camera , 1993, Applications of Invariance in Computer Vision.

[38]  Olivier D. Faugeras,et al.  Determining the fundamental matrix with planes: instability and new algorithms , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Amnon Shashua,et al.  On Geomatric and Algebraic Aspects of 3D Affine and Projective Structures from Perspective 2D Views , 1993, Applications of Invariance in Computer Vision.

[40]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[41]  Jean Ponce,et al.  Analytical Methods for Uncalibrated Stereo and Motion Reconstruction , 1994, ECCV.

[42]  Paul A. Beardsley,et al.  Navigation using Affine Structure from Motion , 1994, ECCV.

[43]  Ian D. Reid,et al.  Recursive Affine Structure and Motion from Image Sequences , 1994, ECCV.

[44]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, ECCV.

[45]  Richard I. Hartley,et al.  Projective reconstruction from line correspondences , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Richard I. Hartley,et al.  An algorithm for self calibration from several views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Michel Dhome,et al.  Camera Calibration From Spheres Images , 1994, ECCV.

[48]  Andrew Zisserman,et al.  Motion From Point Matches Using Affine Epipolar Geometry , 1994, ECCV.

[49]  Thierry Viéville,et al.  Auto-calibration of visual sensor parameters on a robotic head , 1994, Image Vis. Comput..

[50]  Nassir Navab,et al.  Relative affine structure: theory and application to 3D reconstruction from perspective views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Olivier D. Faugeras,et al.  An optimization framework for efficient self-calibration and motion determination , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[52]  Rachid Deriche,et al.  Robust Recovery of the Epipolar Geometry for an Uncalibrated Stereo Rig , 1994, ECCV.

[53]  Richard I. Hartley.  Self-Calibration from Multiple Views with a Rotating Camera , 1994, ECCV.

[54]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[55]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1995, Int. J. Robotics Res..

[56]  Olivier D. Faugeras,et al.  Camera Calibration, Scene Motion and Structure recovery from point correspondences and fundamental m , 1997 .