On Computing Metric Upgrades of Projective Reconstructions Under the Rectangular Pixel Assumption

This paper shows how to upgrade the projective reconstruction of a scene to a metric one in the case where the only assumption made about the cameras observing that scene is that they have rectangular pixels (zero-skew cameras). The proposed approach is based on a simple characterization of zero-skew projection matrices in terms of line geometry, and it handles zero-skew cameras with arbitrary or known aspect ratios in a unified framework. The metric upgrade computation is decomposed into a sequence of linear operations, including linear least-squares parameter estimation and eigenvalue-based symmetric matrix factorization, followed by an optional non-linear least-squares refinement step. A few classes of critical motions for which a unique solution cannot be found are spelled out. A MATLAB implementation has been constructed and preliminary experiments with real data are presented.

[1]  M. Pollefeys.  Self-calibration and metric 3d reconstruction from uncalibrated image sequences , 1999 .

[2]  Manolis I. A. Lourakis,et al.  Estimating the Jacobian of the Singular Value Decomposition: Theory and Applications , 2000, ECCV.

[3]  A. Heyden.  Geometry and algebra of multiple projective transformations , 1995 .

[4]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[5]  O. Faugeras.  Strati cation of D vision projective a ne and metric representations , 1995 .

[6]  Anders Heyden,et al.  Flexible calibration: minimal cases for auto-calibration , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Richard I. Hartley,et al.  An algorithm for self calibration from several views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[9]  Thomas S. Huang,et al.  Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1995, Int. J. Robotics Res..

[11]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[13]  Amnon Shashua,et al.  Projective depth: A geometric invariant for 3D reconstruction from two perspective/orthographic views and for visual recognition , 1993, 1993 (4th) International Conference on Computer Vision.

[14]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[15]  T. Kanade,et al.  A multi-body factorization method for motion analysis , 1995, ICCV 1995.

[16]  Andrew W. Fitzgibbon,et al.  Automatic 3D model acquisition and generation of new images from video sequences , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[17]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[19]  Roger Mohr,et al.  Euclidean constraints for uncalibrated reconstruction , 1993, 1993 (4th) International Conference on Computer Vision.

[20]  Elizabeth Eskow,et al.  A New Modified Cholesky Factorization , 1990, SIAM J. Sci. Comput..

[21]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[22]  Philip E. Gill,et al.  Newton-type methods for unconstrained and linearly constrained optimization , 1974, Math. Program..

[23]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[26]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[27]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[28]  Daphna Weinshall,et al.  Linear and incremental acquisition of invariant shape models from image sequences , 1993, 1993 (4th) International Conference on Computer Vision.

[29]  O. Faugeras,et al.  Grassman–Cayley algebra for modelling systems of cameras and the algebraic equations of the manifold of trifocal tensors , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[30]  Richard I. Hartley,et al.  In defence of the 8-point algorithm , 1995, Proceedings of IEEE International Conference on Computer Vision.

[31]  Anders Heyden,et al.  Minimal Conditions on Intrinsic Parameters for Euclidean Reconstruction , 1998, ACCV.

[32]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[33]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[34]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.