Distributed Very Large Scale Bundle Adjustment by Global Camera Consensus

The increasing scale of Structure-from-Motion is fundamentally limited by the conventional optimization framework for the all-in-one global bundle adjustment. In this paper, we propose a distributed approach to coping with this global bundle adjustment for very large scale Structure-from-Motion computation. First, we derive the distributed formulation from the classical optimization algorithm ADMM, Alternating Direction Method of Multipliers, based on the global camera consensus. Then, we analyze the conditions under which the convergence of this distributed optimization would be guaranteed. In particular, we adopt over-relaxation and self-adaption schemes to improve the convergence rate. After that, we propose to split the large scale camera-point visibility graph in order to reduce the communication overheads of the distributed computing. The experiments on both public large scale SfM data-sets and our very large scale aerial photo sets demonstrate that the proposed distributed method clearly outperforms the state-of-the-art method in efficiency and accuracy.

[1]  Anders P. Eriksson,et al.  A Consensus-Based Framework for Distributed Bundle Adjustment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jonathan Eckstein.  Parallel alternating direction multiplier decomposition of convex programs , 1994 .

[3]  Frank Dellaert,et al.  Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing , 2006, Int. J. Robotics Res..

[4]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[6]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  David Martin,et al.  Street View Motion-from-Structure-from-Motion , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Michael C. Ferris,et al.  Operator-Splitting Methods for Monotone Affine Variational Inequalities, with a Parallel Application to Optimal Control , 1998, INFORMS J. Comput..

[9]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Richard Szeliski,et al.  Bundle Adjustment in the Large , 2010, ECCV.

[11]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[12]  Sameer Agarwal,et al.  Visibility Based Preconditioning for bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Richard I. Hartley,et al.  Projective least-squares: Global solutions with local optimization , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Jan-Michael Frahm,et al.  Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset) , 2015, CVPR 2015.

[15]  Lei Zhou,et al.  Accurate, Scalable and Parallel Structure from Motion , 2017, ArXiv.

[16]  Jan-Michael Frahm,et al.  From single image query to detailed 3D reconstruction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  B. He,et al.  Alternating Direction Method with Self-Adaptive Penalty Parameters for Monotone Variational Inequalities , 2000 .

[18]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[19]  Noah Snavely,et al.  Robust Global Translations with 1DSfM , 2014, ECCV.

[20]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[21]  Frank Dellaert,et al.  Generalized subgraph preconditioners for large-scale bundle adjustment , 2011, Vision.

[22]  Jianxiong Xiao,et al.  Local Readjustment for High-Resolution 3D Reconstruction , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Long Quan,et al.  Joint Camera Clustering and Surface Segmentation for Large-Scale Multi-view Stereo , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[26]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[27]  M. Fukushima,et al.  A generalized proximal point algorithm for certain non-convex minimization problems , 1981 .

[28]  Frank Dellaert,et al.  Out-of-Core Bundle Adjustment for Large-Scale 3D Reconstruction , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[29]  Ping Tan,et al.  A Global Linear Method for Camera Pose Registration , 2013, 2013 IEEE International Conference on Computer Vision.

[30]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[31]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[32]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[33]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2015, ICASSP.

[34]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[35]  Anthony J. Yezzi,et al.  A Compact Formula for the Derivative of a 3-D Rotation in Exponential Coordinates , 2013, Journal of Mathematical Imaging and Vision.

[36]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[37]  Long Quan,et al.  Graph-Based Consistent Matching for Structure-from-Motion , 2016, ECCV.

[38]  Frank Dellaert,et al.  Mining Structure Fragments for Smart Bundle Adjustment , 2014, BMVC.

[39]  Xiaojun Chen,et al.  Alternating Direction Method of Multipliers for a Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction , 2015, SIAM J. Imaging Sci..

[40]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[41]  Alexander Kaplan,et al.  Proximal Point Methods and Nonconvex Optimization , 1998, J. Glob. Optim..

[42]  Long Quan,et al.  Resampling Structure from Motion , 2010, ECCV.