MultiCol Bundle Adjustment: A Generic Method for Pose Estimation, Simultaneous Self-Calibration and Reconstruction for Arbitrary Multi-Camera Systems

In this paper, we present a generic, modular bundle adjustment method for pose estimation, simultaneous self-calibration and reconstruction for multi-camera systems. In contrast to other approaches that use bearing vectors (camera rays) as observations, we extend the common collinearity equations with a general camera model and include the relative orientation of each camera w.r.t to the fixed multi-camera system frame yielding the extended collinearity equations that directly express all image observations as functions of all unknowns. Hence, we can either calibrate the camera system, the cameras, reconstruct the observed scene, and/or simply estimate the pose of the system by including the corresponding parameter block into the Jacobian matrix. Apart from evaluating the implementation with comprehensive simulations, we benchmark our method against recently published methods for pose estimation and bundle adjustment for multi-camera systems. Finally, all methods are evaluated using a 6 degree of freedom ground truth data set, that was recorded with a lasertracker.

[1]  Kurt Konolige,et al.  g 2 o: A general Framework for (Hyper) Graph Optimization , 2011 .

[2]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[3]  P. Scott.  CLOSE RANGE CAMERA CALIBRATION: A NEW METHOD , 2006 .

[4]  Joachim Denzler,et al.  Intrinsic and extrinsic active self-calibration of multi-camera systems , 2013, Machine Vision and Applications.

[5]  Hugh F. Durrant-Whyte,et al.  Camera Calibration for Miniature, Low-cost, Wide-angle Imaging Systems , 2007, BMVC.

[6]  Yolanda González Cid,et al.  Real-time 3d SLAM with wide-angle vision , 2004 .

[7]  Patrick Rives,et al.  Single View Point Omnidirectional Camera Calibration from Planar Grids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[8]  Pascal Vasseur,et al.  Extrinsic calibration of heterogeneous cameras by line images , 2014, Machine Vision and Applications.

[9]  Jan-Michael Frahm,et al.  Simple calibration of non-overlapping cameras with a mirror , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Andrew J. Davison,et al.  SLAM-based automatic extrinsic calibration of a multi-camera rig , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[14]  Wolfgang Förstner,et al.  Minimal Representations for Uncertainty and Estimation in Projective Spaces , 2010, ACCV.

[15]  Roland Siegwart,et al.  A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation , 2011, CVPR 2011.

[16]  Roland Siegwart,et al.  A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[17]  Yubin Kuang,et al.  Revisiting the PnP Problem: A Fast, General and Optimal Solution , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Stefan Hinz,et al.  Improved wide-angle, fisheye and omnidirectional camera calibration , 2015 .

[19]  Joel A. Hesch,et al.  A Direct Least-Squares (DLS) method for PnP , 2011, 2011 International Conference on Computer Vision.

[20]  Luc Van Gool,et al.  Generalised Linear Pose Estimation , 2007, BMVC.

[21]  Hans-Gerd Maas.  Image sequence based automatic multi-camera system calibration techniques 1 Revised version of a pap , 1999 .

[22]  Kostas Daniilidis,et al.  A Unifying Theory for Central Panoramic Systems and Practical Applications , 2000, ECCV.

[23]  Yiannis Aloimonos,et al.  Calibration of a Multicamera Network , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[24]  Davide Scaramuzza,et al.  Omnidirectional Camera , 2014, Computer Vision, A Reference Guide.

[25]  Tomáš Svoboda.  Quick Guide to Multi-Camera Self-Calibration , 2003 .

[26]  Roland Siegwart,et al.  Using multi-camera systems in robotics: Efficient solutions to the NPnP problem , 2013, 2013 IEEE International Conference on Robotics and Automation.

[27]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[28]  Marc Pollefeys,et al.  A multiple-camera system calibration toolbox using a feature descriptor-based calibration pattern , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[30]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[31]  Wolfgang Förstner,et al.  Bundle Adjustment and System Calibration with Points at Infinity for Omnidirectional Camera Systems Bündelausgleichung und Systemkalibrierung mit Punkten im Unendlichen für omnidirektionale Kamerasysteme , 2013 .

[32]  Wolfgang Förstner,et al.  Bundle Adjustment for Multi-Camera Systems with Points at Infinity , 2012 .

[33]  Kostas Daniilidis,et al.  Wide Area Multiple Camera Calibration and Estimation of Radial Distortion , 2004 .

[34]  Reinhard Koch,et al.  Pose Estimation for Multi-camera Systems , 2004, DAGM-Symposium.

[35]  Jan-Michael Frahm,et al.  Robust 6DOF Motion Estimation for Non-Overlapping, Multi-Camera Systems , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[36]  Roland Siegwart,et al.  A Toolbox for Easily Calibrating Omnidirectional Cameras , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Juho Kannala,et al.  A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Christian Heipke,et al.  Estimating the Mutual Orientation in a Multi-camera System with a Non Overlapping Field of View , 2011, PIA.

[39]  Francisco Angel Moreno,et al.  The Málaga urban dataset: High-rate stereo and LiDAR in a realistic urban scenario , 2014, Int. J. Robotics Res..

[40]  Hongdong Li,et al.  UPnP: An Optimal O(n) Solution to the Absolute Pose Problem with Universal Applicability , 2014, ECCV.

[41]  Andreas Geiger,et al.  Calibrating and centering quasi-central catadioptric cameras , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[42]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[43]  Luis Puig,et al.  Calibration of omnidirectional cameras in practice: A comparison of methods , 2012, Comput. Vis. Image Underst..

[44]  Giulio Fontana,et al.  Rawseeds ground truth collection systems for indoor self-localization and mapping , 2009, Auton. Robots.

[45]  Andreas Geiger,et al.  Omnidirectional 3D reconstruction in augmented Manhattan worlds , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[48]  Quan Pan,et al.  Pose estimation for multi-camera systems , 2017, 2017 IEEE International Conference on Unmanned Systems (ICUS).

[49]  Laurent Kneip,et al.  OpenGV: A unified and generalized approach to real-time calibrated geometric vision , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[50]  Axel Pinz,et al.  Globally Optimal O(n) Solution to the PnP Problem for General Camera Models , 2008, BMVC.

[51]  Zeyu Li,et al.  Wide-area external multi-camera calibration using vision graphs and virtual calibration object , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[52]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[53]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[54]  Y. Aloimonos,et al.  Complete calibration of a multi-camera network , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).