View Planning and Refractive Modeling for Structure and Motion

This thesis presents contributions to structure-and-motion estimation, a central topic in the field of geometric computer vision. In particular, the problem of view planning is considered, and continuous and discrete optimization-based algorithms are given for how to plan the path of a sensor to its destination, while balancing the competing goals of path length and reconstruction accuracy. The same concepts are then applied to the problem of sequential 3D reconstruction from unordered image sequences. By propagating reconstruction uncertainties and actively selecting the order in which images are used via view planning, significant gains in robustness and computational efficiency are achieved. The second topic of the thesis is refractive structure-and-motion, specifically the problem of absolute pose estimation when the camera and structure are separated by an optically refracting plane. Using methods from algebraic geometry for solving multivariate polynomial systems, efficient minimal and near-minimal solvers are constructed. Finally, a practical method for calibrating a set of cameras under refraction is given, including an algorithm for efficient refractive bundle adjustment.

[1]  Anders Heyden,et al.  A step towards self-calibration in SLAM: Weakly calibrated on-line structure and motion estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[2]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[3]  Richard Szeliski,et al.  Bundle Adjustment in the Large , 2010, ECCV.

[4]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .

[5]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[6]  Zuzana Kukelova,et al.  Polynomial Eigenvalue Solutions to Minimal Problems in Computer Vision , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Kostas Daniilidis,et al.  Optimizing polynomial solvers for minimal geometry problems , 2011, 2011 International Conference on Computer Vision.

[8]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[9]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Hongdong Li,et al.  Five-Point Motion Estimation Made Easy , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[12]  Hongdong Li.  A Simple Solution to the Six-Point Two-View Focal-Length Problem , 2006, ECCV.

[13]  Lawrence J. Schmitt,et al.  Performance characteristics of alternative genetic algorithmic approaches to the traveling salesman problem using path representation: An empirical study , 1998, Eur. J. Oper. Res..

[14]  In-Chan Choi,et al.  A genetic algorithm with a mixed region search for the asymmetric traveling salesman problem , 2003, Comput. Oper. Res..

[15]  Jan-Michael Frahm,et al.  Developing visual sensing strategies through next best view planning , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Zuzana Kukelova,et al.  New Efficient Solution to the Absolute Pose Problem for Camera with Unknown Focal Length and Radial Distortion , 2010, ACCV.

[17]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[18]  Kenichi Kanatani,et al.  Triangulation from Two Views Revisited: Hartley-Sturm vs. Optimal Correction , 2008, BMVC.

[19]  Olivier D. Faugeras,et al.  Automatic calibration and removal of distortion from scenes of structured environments , 1995, Optics & Photonics.

[20]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Sameer Agarwal,et al.  Visibility Based Preconditioning for bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[23]  Martin Byröd,et al.  A Column-Pivoting Based Strategy for Monomial Ordering in Numerical Gröbner Basis Calculations , 2008, ECCV.

[24]  Didier Henrion,et al.  Globally Optimal Estimates for Geometric Reconstruction Problems , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[25]  Dorian Gálvez-López,et al.  CI-Graph simultaneous localization and mapping for three-dimensional reconstruction of large and complex environments using a multicamera system , 2010, J. Field Robotics.

[26]  David A. Cox,et al.  Ideals, Varieties, and Algorithms , 1997 .

[27]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Y.Y. Schechner,et al.  Flat refractive geometry , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[31]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[32]  Carme Torras,et al.  Information-Gain View Planning for Free-Form Object Reconstruction with a 3D ToF Camera , 2012, ACIVS.

[33]  Michel Dhome,et al.  Generic and real-time structure from motion using local bundle adjustment , 2009, Image Vis. Comput..

[34]  Refractive Structure-from-Motion on Underwater Images , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[36]  Alicia Dickenstein,et al.  Solving Polynomial Equations: Foundations, Algorithms, and Applications , 2010 .

[37]  A. Lastra,et al.  An Adaptive Hierarchical Next-Best-View Algorithm for 3D Reconstruction of Indoor Scenes , 2006 .

[38]  Daniel Lazard,et al.  Resolution des Systemes d'Equations Algebriques , 1981, Theor. Comput. Sci..

[39]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[40]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[41]  Geoffrey A. Hollinger,et al.  Uncertainty-driven view planning for underwater inspection , 2012, 2012 IEEE International Conference on Robotics and Automation.

[42]  Jiri Matas,et al.  Randomized RANSAC with Td, d test , 2004, Image Vis. Comput..

[43]  Enrique Dunn,et al.  Parisian camera placement for vision metrology , 2006, Pattern Recognit. Lett..

[44]  Y.F. Li,et al.  Automatic sensor placement for model-based robot vision , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[45]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.

[46]  David Nistér,et al.  A Minimal Solution to the Generalised 3-Point Pose Problem , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[47]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[49]  Visesh Chari,et al.  A theory of multi-layer flat refractive geometry , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Jean Charles Faugère,et al.  A new efficient algorithm for computing Gröbner bases without reduction to zero (F5) , 2002, ISSAC '02.

[51]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[52]  Hongdong Li,et al.  Motion Estimation for Nonoverlapping Multicamera Rigs: Linear Algebraic and L∞ Geometric Solutions , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Mansoor Davoodi Monfared,et al.  Multi-objective path planning in discrete space , 2013, Appl. Soft Comput..

[54]  R. Hartley.  Triangulation, Computer Vision and Image Understanding , 1997 .

[55]  R. Hartley,et al.  Multiple-View Geometry under the L 1-Norm , 2007 .

[56]  Karl Johan Åström,et al.  Constraint Enforcement in Structure and Motion Applied to Closing and Open Sequence , 2004 .

[57]  M. Er.  Quadratic optimization problems in robust beamforming , 1990 .

[58]  Yubin Kuang,et al.  Exploiting p-fold symmetries for faster polynomial equation solving , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[59]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[60]  Karl Johan Åström,et al.  Solutions to Minimal Generalized Relative Pose Problems , 2005 .

[61]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[62]  Anne Jordt,et al.  Refractive Calibration of Underwater Cameras , 2012, ECCV.

[63]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[64]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[65]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[66]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[67]  Erik Ask.  Methods for Optimal Model Fitting and Sensor Calibration , 2014 .

[68]  Minglun Gong,et al.  Underwater Camera Calibration Using Wavelength Triangulation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Tsuhan Chen,et al.  Multi-view 3D reconstruction for scenes under the refractive plane with known vertical direction , 2011, 2011 International Conference on Computer Vision.

[70]  Joachim Denzler,et al.  Online Next-Best-View Planning for Accuracy Optimization Using an Extended E-Criterion , 2010, 2010 20th International Conference on Pattern Recognition.

[71]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[72]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[73]  Clive S. Fraser.  Network design considerations for non-topographic photogrammetry , 1994 .

[74]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[75]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[76]  H. Schröcker,et al.  Re(cid:176)ections on Refractions , 2000 .

[77]  J. Lasserre.  Convergent LMI relaxations for nonconvex quadratic programs , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[78]  Martin Byröd,et al.  Fast and Stable Polynomial Equation Solving and Its Application to Computer Vision , 2009, International Journal of Computer Vision.

[79]  Yubin Kuang,et al.  Numerically Stable Optimization of Polynomial Solvers for Minimal Problems , 2012, ECCV.

[80]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[81]  J. Kiefer,et al.  Sequential minimax search for a maximum , 1953 .

[82]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[83]  Jan Verschelde,et al.  Algorithm 795: PHCpack: a general-purpose solver for polynomial systems by homotopy continuation , 1999, TOMS.

[84]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[85]  Andreas Krause,et al.  Adaptive Submodularity: A New Approach to Active Learning and Stochastic Optimization , 2010, COLT 2010.

[86]  N. Madras,et al.  THE SELF-AVOIDING WALK , 2006 .

[87]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[88]  Visesh Chari,et al.  Multi-View Geometry of the Refractive Plane , 2009, BMVC.

[89]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[90]  Yee-Hong Yang,et al.  Two-View Underwater Structure and Motion for Cameras under Flat Refractive Interfaces , 2012, ECCV.

[91]  S. Mason.  Heuristic reasoning strategy for automated sensor placement , 1997 .

[92]  Adi Shamir,et al.  Efficient Algorithms for Solving Overdefined Systems of Multivariate Polynomial Equations , 2000, EUROCRYPT.

[93]  Peter K. Allen,et al.  Data acquisition and view planning for 3-D modeling tasks , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[94]  H. Singh,et al.  Hemispherical refraction and camera calibration in underwater vision , 2008, OCEANS 2008.

[95]  Martin Byröd,et al.  Conjugate Gradient Bundle Adjustment , 2010, ECCV.

[96]  Shengyong Chen,et al.  Active Sensor Planning for Multiview Vision Tasks , 2008 .

[97]  Hongdong Li,et al.  Rotation Averaging , 2013, International Journal of Computer Vision.

[98]  Einari Kilpelä,et al.  Compensation of systematic errors of image and model coordinates , 1981 .

[99]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[100]  Carl Olsson,et al.  Stable Structure from Motion for Unordered Image Collections , 2011, SCIA.

[101]  Zuzana Kukelova,et al.  Closed-Form Solutions to Minimal Absolute Pose Problems with Known Vertical Direction , 2010, ACCV.

[102]  Andrew Zisserman,et al.  Multiple View Geometry , 1999 .

[103]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[104]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[105]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[106]  Rafael Murrieta-Cid,et al.  Hierarchical Ray Tracing for Fast Volumetric Next-Best-View Planning , 2013, 2013 International Conference on Computer and Robot Vision.

[107]  Magnus Oskarsson,et al.  Automatic geometric reasoning in structure and motion estimatio , 2000, Pattern Recognit. Lett..

[108]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[109]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..