Adding Synchronization and Rolling Shutter in Multi-Camera Bundle Adjustment

Multi-cameras built by fixing together several consumer cameras become popular and are convenient for applications like 360 videos. However, their self-calibration is not easy since they are composed of several unsynchronized and rolling shutter cameras. This paper introduces a new bundle adjustment for these multi-cameras that estimates not only the usual parameters (camera poses and 3D points) but also the synchronization and the rolling shutter of the cameras. We experiment using videos taken by GoPro cameras mounted on a helmet, moving along trajectories of several hundreds of meters or kilometers, and compare our results to ground truth.

[1]  Michel Dhome,et al.  Generic and Real-Time Structure from Motion , 2007, BMVC.

[2]  Daniele Mortari,et al.  How to avoid singularity when using Euler angles , 2005 .

[3]  Roland Siegwart,et al.  Unified temporal and spatial calibration for multi-sensor systems , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  David Martin,et al.  Street View Motion-from-Structure-from-Motion , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[6]  Johannes Schneider,et al.  Bundle Adjustment and System Calibration with Points at Infinity for Omnidirectional Cameras , 2013 .

[7]  S. Shankar Sastry,et al.  Geometric Models of Rolling-Shutter Cameras , 2005, ArXiv.

[8]  Michael Felsberg,et al.  Rolling shutter bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Gabe Sibley,et al.  Spline Fusion: A continuous-time representation for visual-inertial fusion with application to rolling shutter cameras , 2013, BMVC.

[10]  Michel Dhome,et al.  Fast calibration of embedded non-overlapping cameras , 2011, 2011 IEEE International Conference on Robotics and Automation.

[11]  Jean-Marc Lavest,et al.  Multiple View 3D Reconstruction with Rolling Shutter Cameras , 2015, VISIGRAPP.

[12]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[13]  Roland Siegwart,et al.  Rolling Shutter Camera Calibration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.