Synchronization and Self-Calibration for Helmet-Held Consumer Cameras, Applications to Immersive 3D Modeling and 360 Video

This paper presents the first 3D reconstruction system using unsynchronized and helmet-held consumer cameras, without the use of a calibration pattern. Our assumptions are easy to meet in practice: the cameras have the same setting (frequency, image resolution, field-of-view, roughly equiangular). First, the time offsets between cameras are estimated without accurate calibration as input. Second, both inter-camera rotations and intrinsic parameters are refined using structure-from-motion and bundle adjustment. We experiment both synchronization and self-calibration on four GoPro cameras mounted on a helmet, such that the resulting multi-camera is assumed to be central and provides a 360 degree field-of-view in the horizontal plane. A surface is also estimated from a multi-camera video acquired by walking in a city.

[1]  Mubarak Shah,et al.  TEMPORAL SYNCHRONIZATION FROM CAMERA MOTION , 2003 .

[2]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Roland Siegwart,et al.  Unified temporal and spatial calibration for multi-sensor systems , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Michel Dhome,et al.  Do We Really Need an Accurate Calibration Pattern to Achieve a Reliable Camera Calibration? , 1998, ECCV.

[5]  Long Quan,et al.  Match Propagation for Image-Based Modeling and Rendering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Michal Havlena,et al.  From Google Street View to 3D city models , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[7]  Richard Szeliski,et al.  Symmetric Sub-Pixel Stereo Matching , 2002, ECCV.

[8]  C. Zach,et al.  Generalized Detection and Merging of Loop Closures for Video Sequences , 2008 .

[9]  Andrew J. Davison,et al.  SLAM-based automatic extrinsic calibration of a multi-camera rig , 2011, 2011 IEEE International Conference on Robotics and Automation.

[10]  Nick Barnes,et al.  Rotation Averaging with Application to Camera-Rig Calibration , 2009, ACCV.

[11]  Maxime Lhuillier,et al.  Incremental Solid Modeling from Sparse Structure-from-Motion Data with Improved Visual Artifacts Removal , 2014, 2014 22nd International Conference on Pattern Recognition.

[12]  Johannes Schneider,et al.  Bundle Adjustment and System Calibration with Points at Infinity for Omnidirectional Cameras , 2013 .

[13]  Maxime Lhuillier,et al.  Manifold surface reconstruction of an environment from sparse Structure-from-Motion data , 2013, Comput. Vis. Image Underst..

[14]  Michel Dhome,et al.  Fast calibration of embedded non-overlapping cameras , 2011, 2011 IEEE International Conference on Robotics and Automation.

[15]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[16]  Michel Dhome,et al.  Generic and Real-Time Structure from Motion , 2007, BMVC.

[17]  Simone Gasparini,et al.  Camera Models and Fundamental Concepts Used in Geometric Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[18]  Reinhard Koch,et al.  Calibration of a Multi-camera Rig from Non-overlapping Views , 2007, DAGM-Symposium.

[19]  Paulo Jorge Ramalho Oliveira,et al.  Synchronization of Two Independently Moving Cameras without Feature Correspondences , 2014, ECCV.

[20]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[21]  Ian D. Reid,et al.  Self-Calibration of Rotating and Zooming Cameras , 2002, International Journal of Computer Vision.