A perspective factorization method for Euclidean reconstruction with uncalibrated cameras

Structure from motion (SFM), which is recovering camera motion and scene structure from image sequences, has various applications, such as scene modelling, robot navigation, object recognition and virtual reality. Most of previous research on SFM requires the use of intrinsically calibrated cameras. In this paper we describe a factorization-based method to recover Euclidean structure from multiple perspective views with uncalibrated cameras. The method first performs a projective reconstruction using a bilinear factorization algorithm, and then converts the projective solution to a Euclidean one by enforcing metric constraints. The process of updating a projective solution to a full metric one is referred as normalization in most factorization-based SFM methods. We present three normalization algorithms which enforce Euclidean constraints on camera calibration parameters to recover the scene structure and the camera calibration simultaneously, assuming zero skew cameras. The first two algorithms are linear, one for dealing with the case that only the focal lengths are unknown, and another for the case that the focal lengths and the constant principal point are unknown. The third algorithm is bilinear, dealing with the case that the focal lengths, the principal points and the aspect ratios are all unknown. The results of experiments are presented. Copyright © 2002 John Wiley & Sons, Ltd.

[1]  Anders Heyden,et al.  Euclidean reconstruction from image sequences with varying and unknown focal length and principal point , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[3]  Richard I. Hartley,et al.  Linear self-calibration of a rotating and zooming camera , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1995, Int. J. Robotics Res..

[5]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[6]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  David W. Jacobs,et al.  Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[9]  Anders Heyden,et al.  An iterative factorization method for projective structure and motion from image sequences , 1999, Image Vis. Comput..

[10]  Richard I. Hartley,et al.  Computation of the Quadrifocal Tensor , 1998, ECCV.

[11]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[12]  Stéphane Christy,et al.  Euclidean Reconstruction: From Paraperspective to Perspective , 1996, ECCV.

[13]  A. Heyden,et al.  Euclidean reconstruction from constant intrinsic parameters , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[14]  Long Quan,et al.  Self-calibration of an affine camera from multiple views , 1996, International Journal of Computer Vision.

[15]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Hua Yu,et al.  3D shape and motion by SVD under higher-order approximation of perspective projection , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[17]  Stéphane Christy,et al.  Euclidean Shape and Motion from Multiple Perspective Views by Affine Iterations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[19]  Peter F. Sturm,et al.  Algorithms for plane-based pose estimation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Anders Heyden.  Reduced Multilinear Constraints: Theory and Experiments , 2004, International Journal of Computer Vision.

[21]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[22]  C. Rother,et al.  Linear multi view reconstruction and camera recovery , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[25]  Bill Triggs,et al.  Critical Motions for Auto-Calibration When Some Intrinsic Parameters Can Vary , 2000, Journal of Mathematical Imaging and Vision.

[26]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Václav Hlavác,et al.  Projective Reconstruction from N Views Having One View in Common , 1999, Workshop on Vision Algorithms.

[28]  Martial Hebert,et al.  Iterative projective reconstruction from multiple views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[29]  Bill Triggs,et al.  Matching constraints and the joint image , 1995, Proceedings of IEEE International Conference on Computer Vision.

[30]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[31]  Jean Ponce,et al.  On Computing Metric Upgrades of Projective Reconstructions Under the Rectangular Pixel Assumption , 2000, SMILE.

[32]  Long Quan,et al.  Invariants of Six Points and Projective Reconstruction From Three Uncalibrated Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Amnon Shashua,et al.  The Rank 4 Constraint in Multiple (>=3) View Geometry , 1996, ECCV.

[35]  Daphna Weinshall,et al.  Dual Computation of Projective Shape and Camera Positions from Multiple Images , 1998, International Journal of Computer Vision.

[36]  Peter Sturm.  Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length , 2002, Image Vis. Comput..