Large Scale Photometric Bundle Adjustment

Direct methods have shown promise on visual odometry and SLAM, leading to greater accuracy and robustness over feature-based methods. However, offline 3-d reconstruction from internet images has not yet benefited from a joint, photometric optimization over dense geometry and camera parameters. Issues such as the lack of brightness constancy, and the sheer volume of data, make this a more challenging task. This work presents a framework for jointly optimizing millions of scene points and hundreds of camera poses and intrinsics, using a photometric cost that is invariant to local lighting changes. The improvement in metric reconstruction accuracy that it confers over feature-based bundle adjustment is demonstrated on the large-scale Tanks & Temples benchmark. We further demonstrate qualitative reconstruction improvements on an internet photo collection, with challenging diversity in lighting and camera intrinsics.

[1]  Gene H. Golub,et al.  The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate , 1972, Milestones in Matrix Computation.

[2]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[3]  Julien Lefèvre,et al.  An Exact Formula for Calculating Inverse Radial Lens Distortions , 2016, Sensors.

[4]  ARNO KNAPITSCH,et al.  Tanks and temples , 2017, ACM Trans. Graph..

[5]  Oliver J. Woodford.  Using Normalized Cross Correlation in Least Squares Optimizations , 2018, ArXiv.

[6]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[7]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[8]  Ronen Basri,et al.  A Survey on Structure from Motion , 2017, ArXiv.

[9]  Kaj Madsen,et al.  Methods for Non-Linear Least Squares Problems , 1999 .

[10]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jean Ponce,et al.  Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Marc Pollefeys,et al.  Photometric Bundle Adjustment for Dense Multi-view 3D Modeling , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Daniel Cremers,et al.  A Superresolution Framework for High-Accuracy Multiview Reconstruction , 2009, DAGM-Symposium.

[14]  M. Brooks,et al.  Recovering unknown focal lengths in self-calibration: an essentially linear algorithm and degenerate configurations , 1996 .

[15]  Luo Si,et al.  General, Nested, and Constrained Wiberg Minimization , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[17]  Leif Kobbelt,et al.  Iterative multi - view plane fitting , 2006 .

[18]  Brett Browning,et al.  Direct Visual Odometry in Low Light Using Binary Descriptors , 2017, IEEE Robotics and Automation Letters.

[19]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Brett Browning,et al.  Photometric Bundle Adjustment for Vision-Based SLAM , 2016, ACCV.

[22]  Marc Pollefeys,et al.  Illumination change robustness in direct visual SLAM , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Jan-Michael Frahm,et al.  Pixelwise View Selection for Unstructured Multi-View Stereo , 2016, ECCV.

[24]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[25]  Andrew W. Fitzgibbon,et al.  Projective Bundle Adjustment from Arbitrary Initialization Using the Variable Projection Method , 2016, ECCV.

[26]  Daniel Cremers,et al.  A Super-Resolution Framework for High-Accuracy Multiview Reconstruction , 2013, International Journal of Computer Vision.

[27]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[28]  Olaf Kähler,et al.  Tracking and Reconstruction in a Combined Optimization Approach , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.