Camera Motion Estimation by Convex Programming

This thesis studies two inverse problems closely related two each other: The first problem is the estimation of n locations t1, t2, . . . , tn (up to global scale, translation and negation) in R from noisy measurements of a subset of the pairwise lines that connect them, that is, from noisy measurements of ± ti−tj ‖ti−tj‖2 for some pairs (i, j), where the signs are unknown. For the second problem, we assume the availability of (measurements of) the signs, i.e., we consider the estimation of the locations {ti}i=1 (up to global scale and translation) from measurements of the pairwise directions ti−tj ‖ti−tj‖2 for some pairs (i, j). These inverse problems are at the core of the structure from motion (SfM) problem in computer vision, where the ti’s represent camera locations in R . After introducing the inverse problems and providing a discussion of previous related works in Chapter 1, we continue with the characterization of well-posed problem instances in Chapter 2. The contents of Chapter 2 are based on the existing results of parallel rigidity theory, the significance of which was not previously recognized in the context of the camera location estimation problem. Basically, parallel rigidity theory studies the conditions of unique realizability of locations from exact (i.e., noiseless) pairwise lines and directions. We reiterate these results to identify a complete combinatorial characterization of well-posed instances for the two inverse problems, and provide efficient algorithms to decide in the wellposedness of a given instance. In the absence of these conditions, we discuss how to identify maximal subsets of the pairwise measurements inducing well-posed sub-problems. In Chapter 3, we study the inverse problem of location estimation from noisy pairwise line measurements. We firstly present a fundamental difficulty observed for the existing methods, that is, the tendency to produce spurious solutions that are clustered around a few locations. This is a well-known problem in SfM, especially for large, irregular collections of images. To overcome this difficulty, we introduce a semidefinite relaxation (SDR) method, specially tailored to exclude clustering solutions. For this formulation, we prove exact (in the noiseless case) and stable (in the presence of noisy lines) location recovery results. We iii also formulate an alternating direction augmented Lagrangian method (ADM) to efficiently solve the resulting semidefinite program. Chapter 4 investigates the second inverse problem, i.e. estimation of locations from noisy pairwise directions. For current methods in the literature, existence of outliers among the direction measurements typically induces large errors in the location estimates (especially for large, unordered image sets). To reduce the effect of outliers, we introduce two efficient convex programs for robust estimation of locations. As we observe in Chapter 7, compared to the existing alternatives, these methods provide highly accurate location estimates in the presence of outlier direction measurements. Provided with partially corrupted measurements (with sufficiently many noiseless directions), we empirically demonstrate that these programs can even recover the locations exactly. In Chapter 4, we also provide iteratively reweighted least squares (IRLS) solvers in order to efficiently solve these robust convex formulations. To maintain the computational efficiency of the SDR formulation of Chapter 3 and the robust convex programs of Chapter 4 for large sets of locations, we introduce distributed formulations of these solvers in Chapter 5, based on spectral clustering and convex programming. We also show that these distributed methods induce well-posed distributed problem instances. In Chapter 6, we demonstrate how to formulate the camera location estimation problem in terms of the two inverse problems of location estimation (in R) from pairwise lines and directions, and also introduce a convex programming-based method to robustly estimate pairwise lines and directions. Lastly, we demonstrate the utility of our formulations through experiments on synthetic data and real images, in Chapter 7.

[1]  Bill Jackson,et al.  Graph theoretic techniques in the analysis of uniquely localizable sensor networks , 2009 .

[2]  Camillo J. Taylor,et al.  Identifying maximal rigid components in bearing-based localization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[4]  René Vidal,et al.  Distributed image-based 3-D localization of camera sensor networks , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[5]  Jochen Trumpf,et al.  L1 rotation averaging using the Weiszfeld algorithm , 2011, CVPR 2011.

[6]  Bruce Hendrickson,et al.  The Molecule Problem: Exploiting Structure in Global Optimization , 1995, SIAM J. Optim..

[7]  Zhengyou Zhang,et al.  Incremental Motion Estimation Through Local Bundle Adjustment , 2001 .

[8]  Nicolai Bissantz,et al.  Convergence Analysis of Generalized Iteratively Reweighted Least Squares Algorithms on Convex Function Spaces , 2008, SIAM J. Optim..

[9]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[10]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[11]  Amir K. Khandani,et al.  A near maximum likelihood decoding algorithm for MIMO systems based on semi-definite programming , 2005, ISIT.

[12]  Gilad Lerman,et al.  A novel M-estimator for robust PCA , 2011, J. Mach. Learn. Res..

[13]  Willem H. Haemers,et al.  Spectra of Graphs , 2011 .

[14]  Anders P. Eriksson,et al.  Efficient optimization for L∞-problems using pseudoconvexity , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Amit Singer,et al.  Global Registration of Multiple Point Clouds Using Semidefinite Programming , 2013, SIAM J. Optim..

[16]  Ligang Liu,et al.  On affine rigidity , 2010, J. Comput. Geom..

[17]  Yinyu Ye,et al.  Semidefinite programming based algorithms for sensor network localization , 2006, TOSN.

[18]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[19]  Richard I. Hartley,et al.  Recovering Camera Motion Using L\infty Minimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Gábor Pataki,et al.  On the Rank of Extreme Matrices in Semidefinite Programs and the Multiplicity of Optimal Eigenvalues , 1998, Math. Oper. Res..

[21]  Richard Szeliski,et al.  A Multi-stage Linear Approach to Structure from Motion , 2010, ECCV Workshops.

[22]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[23]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Walter Whiteley,et al.  Constraining Plane Configurations in Computer-Aided Design: Combinatorics of Directions and Lengths , 1999, SIAM J. Discret. Math..

[25]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[26]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[27]  B. Anderson,et al.  Sensor and network topologies of formations with direction, bearing, and angle information between agents , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[28]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[29]  G. Laman On graphs and rigidity of plane skeletal structures , 1970 .

[30]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[31]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[32]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[33]  Walter Whiteley,et al.  A matroid on hypergraphs, with applications in scene analysis and geometry , 1989, Discret. Comput. Geom..

[34]  Amit Singer,et al.  Open Problem: Tightness of maximum likelihood semidefinite relaxations , 2014, COLT.

[35]  Ami Wiesel,et al.  Semidefinite relaxation for detection of 16-QAM signaling in MIMO channels , 2005, IEEE Signal Processing Letters.

[36]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[37]  Bastian Katz,et al.  Maximum Rigid Components as Means for Direction-Based Localization in Sensor Networks , 2007, SOFSEM.

[38]  Walter Whiteley,et al.  Some matroids from discrete applied geometry , 1996 .

[39]  Bruce Hendrickson,et al.  Conditions for Unique Graph Realizations , 1992, SIAM J. Comput..

[40]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[41]  Seth J. Teller,et al.  Spectral Solution of Large-Scale Extrinsic Camera Calibration as a Graph Embedding Problem , 2004, ECCV.

[42]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[43]  Richard I. Hartley,et al.  Multiple-View Geometry Under the {$L_\infty$}-Norm , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Brian D. O. Anderson,et al.  A Theory of Network Localization , 2006, IEEE Transactions on Mobile Computing.

[45]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[46]  I. J. Schoenberg Metric spaces and completely monotone functions , 1938 .

[47]  Venu Madhav Govindu,et al.  Combining two-view constraints for motion estimation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[48]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[49]  B. Hendrickson,et al.  Regular ArticleAn Algorithm for Two-Dimensional Rigidity Percolation: The Pebble Game , 1997 .

[50]  Kim-Chuan Toh,et al.  An SDP-Based Divide-and-Conquer Algorithm for Large-Scale Noisy Anchor-Free Graph Realization , 2009, SIAM J. Sci. Comput..

[51]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[52]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[53]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[55]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[56]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[57]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[58]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  David E. Tyler A Distribution-Free $M$-Estimator of Multivariate Scatter , 1987 .

[60]  Zhi-Quan Luo,et al.  Semidefinite Relaxation of Quadratic Optimization Problems , 2010, IEEE Signal Processing Magazine.

[61]  Venu Madhav Govindu Lie-algebraic averaging for globally consistent motion estimation , 2004, CVPR 2004.

[62]  Sanjay Kumar Madria,et al.  Sensor networks: an overview , 2003 .

[63]  Kim-Chuan Toh,et al.  Semidefinite Programming Approaches for Sensor Network Localization With Noisy Distance Measurements , 2006, IEEE Transactions on Automation Science and Engineering.

[64]  Hongyuan Zha,et al.  Spectral Properties of the Alignment Matrices in Manifold Learning , 2009, SIAM Rev..

[65]  Yaron Lipman,et al.  Sensor network localization by eigenvector synchronization over the euclidean group , 2012, TOSN.

[66]  Tomás Pajdla,et al.  Robust Rotation and Translation Estimation in Multiview Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[67]  Anthony Man-Cho So,et al.  Theory of semidefinite programming for Sensor Network Localization , 2005, SODA '05.

[68]  Joel A. Tropp,et al.  Robust computation of linear models, or How to find a needle in a haystack , 2012, ArXiv.

[69]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Ira Kemelmacher-Shlizerman,et al.  Global Motion Estimation from Point Matches , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[71]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[72]  Laurent Jacques,et al.  Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors , 2011, IEEE Transactions on Information Theory.

[73]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[74]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[75]  Amit Singer,et al.  Exact and Stable Recovery of Rotations for Robust Synchronization , 2012, ArXiv.

[76]  T. Eren,et al.  Using Angle of Arrival (Bearing) Information in Network Localization , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[77]  Michal Havlena,et al.  Randomized structure from motion based on atomic 3D models from camera triplets , 2009, CVPR.