Distributed Very Large Scale Bundle Adjustment by Global Camera Consensus

The increasing scale of Structure-from-Motion is fundamentally limited by the conventional optimization framework for the all-in-one global bundle adjustment. In this paper, we propose a distributed approach to coping with this global bundle adjustment for very large scale Structurefrom-Motion computation. First, we derive the distributed formulation from the classical optimization algorithm ADMM, Alternating Direction Method of Multipliers, based on the global camera consensus. Then, we analyze the conditions under which the convergence of this distributed optimization would be guaranteed. In particular, we adopt over-relaxation and self-adaption schemes to improve the convergence rate. After that, we propose to split the large scale camera-point visibility graph in order to reduce the communication overheads of the distributed computing. The experiments on both public large scale SfM data-sets and our very large scale aerial photo sets demonstrate that the proposed distributed method clearly outperforms the state-of-the-art method in efficiency and accuracy.

[1]  Anders P. Eriksson,et al.  A Consensus-Based Framework for Distributed Bundle Adjustment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Noah Snavely,et al.  Robust Global Translations with 1DSfM , 2014, ECCV.

[4]  Jianxiong Xiao,et al.  Local Readjustment for High-Resolution 3D Reconstruction , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Long Quan,et al.  Graph-Based Consistent Matching for Structure-from-Motion , 2016, ECCV.

[6]  Jan-Michael Frahm,et al.  From single image query to detailed 3D reconstruction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  B. He,et al.  Alternating Direction Method with Self-Adaptive Penalty Parameters for Monotone Variational Inequalities , 2000 .

[8]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[9]  Bodo Rosenhahn,et al.  Outdoor and Large-Scale Real-World Scene Analysis , 2012, Lecture Notes in Computer Science.

[10]  Frank Dellaert,et al.  Mining Structure Fragments for Smart Bundle Adjustment , 2014, BMVC.

[11]  Xiaojun Chen,et al.  Alternating Direction Method of Multipliers for a Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction , 2015, SIAM J. Imaging Sci..

[12]  M. Fukushima,et al.  A generalized proximal point algorithm for certain non-convex minimization problems , 1981 .

[13]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[14]  Richard I. Hartley,et al.  Projective least-squares: Global solutions with local optimization , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Jan-Michael Frahm,et al.  Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset) , 2015, CVPR 2015.

[16]  Frank Dellaert,et al.  Generalized subgraph preconditioners for large-scale bundle adjustment , 2011, 2011 International Conference on Computer Vision.

[17]  Lei Zhou,et al.  Accurate, Scalable and Parallel Structure from Motion , 2017, ArXiv.

[18]  Frank Dellaert,et al.  Out-of-Core Bundle Adjustment for Large-Scale 3D Reconstruction , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[20]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[21]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[22]  Alexander Kaplan,et al.  Proximal Point Methods and Nonconvex Optimization , 1998, J. Glob. Optim..

[23]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Sameer Agarwal,et al.  Visibility Based Preconditioning for bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[26]  Frank Dellaert,et al.  Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing , 2006, Int. J. Robotics Res..

[27]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[29]  Long Quan,et al.  Joint Camera Clustering and Surface Segmentation for Large-Scale Multi-view Stereo , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Ping Tan,et al.  Global Structure-from-Motion by Similarity Averaging , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Suvrit Sra,et al.  Scalable nonconvex inexact proximal splitting , 2012, NIPS.

[32]  Ping Tan,et al.  A Global Linear Method for Camera Pose Registration , 2013, 2013 IEEE International Conference on Computer Vision.

[33]  Jonathan Eckstein.  Parallel alternating direction multiplier decomposition of convex programs , 1994 .

[34]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[37]  Anthony J. Yezzi,et al.  A Compact Formula for the Derivative of a 3-D Rotation in Exponential Coordinates , 2013, Journal of Mathematical Imaging and Vision.

[38]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[39]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[40]  Long Quan,et al.  Resampling Structure from Motion , 2010, ECCV.

[41]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Richard Szeliski,et al.  Bundle Adjustment in the Large , 2010, ECCV.

[43]  David Martin,et al.  Street View Motion-from-Structure-from-Motion , 2013, 2013 IEEE International Conference on Computer Vision.

[44]  Michael C. Ferris,et al.  Operator-Splitting Methods for Monotone Affine Variational Inequalities, with a Parallel Application to Optimal Control , 1998, INFORMS J. Comput..

[45]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[46]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .