Globally Optimal Affine and Metric Upgrades in Stratified Autocalibration

We present a practical, stratified autocalibration algorithm with theoretical guarantees of global optimality. Given a projective reconstruction, the first stage of the algorithm upgrades it to affine by estimating the position of the plane at infinity. The plane at infinity is computed by globally minimizing a least squares formulation of the modulus constraints. In the second stage, the algorithm upgrades this affine reconstruction to a metric one by globally minimizing the infinite homography relation to compute the dual image of the absolute conic (DIAC). The positive semidefiniteness of the DIAC is explicitly enforced as part of the optimization process, rather than as a post-processing step. For each stage, we construct and minimize tight convex relaxations of the highly non-convex objective functions in a branch and bound optimization framework. We exploit the problem structure to restrict the search space for the DIAC and the plane at infinity to a small, fixed number of branching dimensions, independent of the number of views. Experimental evidence of the accuracy, speed and scalability of our algorithm is presented on synthetic and real data. MATLAB code for the implementation is made available to the community.

[1]  David Nistér,et al.  Untwisting a Projective Reconstruction , 2004, International Journal of Computer Vision.

[2]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[3]  Luc Van Gool,et al.  Surviving Dominant Planes in Uncalibrated Structure and Motion Recovery , 2002, ECCV.

[4]  Yoram Gat.  A branch-and-bound technique for nano-structure image segmentation , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[5]  Didier Henrion,et al.  GloptiPoly: Global optimization over polynomials with Matlab and SeDuMi , 2003, TOMS.

[6]  David J. Kriegman,et al.  Practical Global Optimization for Multiview Geometry , 2006, ECCV.

[7]  Andrea Fusiello,et al.  Globally convergent autocalibration using interval analysis , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  R. Horst,et al.  Global Optimization: Deterministic Approaches , 1992 .

[9]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[10]  Steven M. Seitz,et al.  Fast algorithms for L∞ problems in multiview geometry , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Anil K. Jain,et al.  Algorithms for feature selection: An evaluation , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Nikolaos V. Sahinidis,et al.  Convex extensions and envelopes of lower semi-continuous functions , 2002, Math. Program..

[14]  David J. Kriegman,et al.  Autocalibration via Rank-Constrained Estimation of the Absolute Quadric , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Richard I. Hartley,et al.  Optimal Algorithms in Multiview Geometry , 2007, ACCV.

[16]  Motilal Agrawal.  On automatic determination of varying focal lengths using semidefinite programming , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[17]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[18]  A. Heyden,et al.  Euclidean reconstruction from constant intrinsic parameters , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[19]  Frederik Schaffalitzky,et al.  Direct Solution of Modulus Constraints , 2000 .

[20]  Ian D. Reid,et al.  Camera calibration and the search for infinity , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[22]  Didier Henrion,et al.  Globally Optimal Estimates for Geometric Reconstruction Problems , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[23]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[24]  Pablo A. Parrilo,et al.  Introducing SOSTOOLS: a general purpose sum of squares programming solver , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[25]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Peter F. Sturm,et al.  A Case Against Kruppa's Equations for Camera Self-Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  James E. Falk,et al.  Jointly Constrained Biconvex Programming , 1983, Math. Oper. Res..

[29]  Luc Van Gool,et al.  Stratified Self-Calibration with the Modulus Constraint , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Charles R. Dyer,et al.  Metric self calibration from screw-transform manifolds , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Carl Olsson,et al.  Branch-and-Bound Methods for Euclidean Registration Problems , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Daniel Freedman.  Effective Tracking through Tree-Search , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Nikolaos V. Sahinidis,et al.  Semidefinite Relaxations of Fractional Programs via Novel Convexification Techniques , 2001, J. Glob. Optim..

[34]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[35]  Thomas M. Breuel.  A Comparison of Search Strategies for Geometric Branch and Bound Algorithms , 2002, ECCV.

[36]  Christoph H. Lampert,et al.  Beyond sliding windows: Object localization by efficient subwindow search , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.