Spline Fusion: A continuous-time representation for visual-inertial fusion with application to rolling shutter cameras

This paper describes a general continuous-time framework for visual-inertial simultaneous localization and mapping and calibration. We show how to use a spline parameterization that closely matches the torque-minimal motion of the sensor. Compared to traditional discrete-time solutions, the continuous-time formulation is particularly useful for solving problems with high-frame rate sensors and multiple unsynchronized devices. We demonstrate the applicability of the method for multi-sensor visual-inertial SLAM and calibration by accurately establishing the relative pose and internal parameters of multiple unsynchronized devices. We also show the advantages of the approach through evaluation and uniform treatment of both global and rolling shutter cameras within visual and visual-inertial SLAM systems.

[1]  Chao Jia,et al.  Probabilistic 3-D motion estimation for rolling shutter video rectification from visual and inertial measurements , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[2]  David W. Murray,et al.  Improving the Agility of Keyframe-Based SLAM , 2008, ECCV.

[3]  M. Cox.  The Numerical Evaluation of B-Splines , 1972 .

[4]  Richard Szeliski,et al.  Removing rolling shutter wobble , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Ian D. Reid,et al.  RSLAM: A System for Large-Scale Mapping in Constant-Time Using Stereo , 2011, International Journal of Computer Vision.

[6]  Patrick Rives,et al.  Accurate Quadrifocal Tracking for Robust 3D Visual Odometry , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[7]  C. D. Boor,et al.  On Calculating B-splines , 1972 .

[8]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[9]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[10]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[11]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Roland Siegwart,et al.  Fusion of IMU and Vision for Absolute Scale Estimation in Monocular SLAM , 2011, J. Intell. Robotic Syst..

[13]  S. Shankar Sastry,et al.  Geometric Models of Rolling-Shutter Cameras , 2005, ArXiv.

[14]  Sung Yong Shin,et al.  A C/sup 2/-continuous B-spline quaternion curve interpolating a given sequence of solid orientations , 1995, Proceedings Computer Animation'95.

[15]  Michael Felsberg,et al.  Rolling shutter bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  P. Crouch,et al.  The De Casteljau Algorithm on Lie Groups and Spheres , 1999 .

[17]  Ian D. Reid,et al.  A hybrid SLAM representation for dynamic marine environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[18]  Kurt Konolige,et al.  Double window optimisation for constant time visual SLAM , 2011, 2011 International Conference on Computer Vision.

[19]  Sung Yong Shin,et al.  A general construction scheme for unit quaternion curves with simple high order derivatives , 1995, SIGGRAPH.

[20]  Stergios I. Roumeliotis,et al.  A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation , 2008, IEEE Transactions on Robotics.

[21]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[22]  Paul Timothy Furgale,et al.  Continuous-time batch estimation using temporal basis functions , 2012, 2012 IEEE International Conference on Robotics and Automation.

[23]  Hauke Strasdat,et al.  Local accuracy and global consistency for efficient SLAM , 2012 .

[24]  A. Vedaldi,et al.  Inertial Structure From Motion with Autocalibration , 2007 .

[25]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[26]  Javier Civera,et al.  Unified Inverse Depth Parametrization for Monocular SLAM , 2006, Robotics: Science and Systems.

[27]  Kaihuai Qin.  General matrix representations for B-splines , 2014, The Visual Computer.

[28]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.