Calibration and Noise Identification of a Rolling Shutter Camera and a Low-Cost Inertial Measurement Unit

A low-cost inertial measurement unit (IMU) and a rolling shutter camera form a conventional device configuration for localization of a mobile platform due to their complementary properties and low costs. This paper proposes a new calibration method that jointly estimates calibration and noise parameters of the low-cost IMU and the rolling shutter camera for effective sensor fusion in which accurate sensor calibration is very critical. Based on the graybox system identification, the proposed method estimates unknown noise density so that we can minimize calibration error and its covariance by using the unscented Kalman filter. Then, we refine the estimated calibration parameters with the estimated noise density in batch manner. Experimental results on synthetic and real data demonstrate the accuracy and stability of the proposed method and show that the proposed method provides consistent results even with unknown noise density of the IMU. Furthermore, a real experiment using a commercial smartphone validates the performance of the proposed calibration method in off-the-shelf devices.

[1]  Jan Skaloud,et al.  Bundle adjustment with raw inertial observations in UAV applications , 2017 .

[2]  Kuk-Jin Yoon,et al.  Robust calibration of an ultralow-cost inertial measurement unit and a camera: Handling of severe system uncertainty , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[4]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[5]  Anastasios I. Mourikis,et al.  High-fidelity sensor modeling and self-calibration in vision-aided inertial navigation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Roland Siegwart,et al.  Maximum Likelihood Identification of Inertial Sensor Noise Model Parameters , 2016, IEEE Sensors Journal.

[7]  Emanuele Menegatti,et al.  A robust and easy to implement method for IMU calibration without external equipments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Richard J. Vaccaro,et al.  Statistical Modeling of Rate Gyros , 2012, IEEE Transactions on Instrumentation and Measurement.

[9]  I. Colomina,et al.  Relative INS/GNSS aerial control in integrated sensor orientation: Models and performance , 2012 .

[10]  Yi Lin,et al.  Autonomous aerial navigation using monocular visual‐inertial fusion , 2018, J. Field Robotics.

[11]  Roland Siegwart,et al.  Robust Real-Time Visual Odometry with a Single Camera and an IMU , 2011, BMVC.

[12]  R. Siegwart,et al.  Self-supervised calibration for robotic systems , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[13]  A. H. Mohamed,et al.  Adaptive Kalman Filtering for INS/GPS , 1999 .

[14]  Robert M. Rogers,et al.  Applied Mathematics in Integrated Navigation Systems , 2000 .

[15]  Roland Siegwart,et al.  Extending kalibr: Calibrating the extrinsics of multiple IMUs and of individual axes , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Robert L. Williams,et al.  Case study: Inertial measurement unit calibration platform , 2000 .

[17]  John J. Leonard,et al.  Towards consistent visual-inertial navigation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Magnus Jansson,et al.  Joint calibration of an inertial measurement unit and coordinate transformation parameters using a monocular camera , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.

[19]  Darius Burschka,et al.  Optimization based IMU camera calibration , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Roland Siegwart,et al.  Rolling Shutter Camera Calibration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Thomas B. Schön,et al.  Modeling and Calibration of Inertial and Vision Sensors , 2010, Int. J. Robotics Res..

[22]  Zuzana Kukelova,et al.  R6P - Rolling Shutter Absolute Camera Pose , 2015, CVPR 2015.

[23]  Rudolph van der Merwe,et al.  Sigma-point kalman filters for probabilistic inference in dynamic state-space models , 2004 .

[24]  Yunhui Liu,et al.  Automatic calibration for inertial measurement unit , 2012, 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV).

[25]  Yi Liu,et al.  Monocular Visual-Inertial SLAM: Continuous Preintegration and Reliable Initialization , 2017, Sensors.

[26]  Peter I. Corke,et al.  Empirical modelling of rolling shutter effect , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[27]  David Jacobs,et al.  CTSR 2011-03 Digital Video Stabilization and Rolling Shutter Correction using Gyroscopes , 2011 .

[28]  Frank van Graas,et al.  Inertial Measurement Unit Calibration Platform , 1999 .

[29]  Anastasios I. Mourikis,et al.  Vision-aided inertial navigation with rolling-shutter cameras , 2014, Int. J. Robotics Res..

[30]  Peter Rockett,et al.  Performance assessment of feature detection algorithms: a methodology and case study on corner detectors , 2003, IEEE Trans. Image Process..

[31]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[32]  Roland Siegwart,et al.  Unified temporal and spatial calibration for multi-sensor systems , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Anastasios I. Mourikis,et al.  Online temporal calibration for camera–IMU systems: Theory and algorithms , 2014, Int. J. Robotics Res..

[34]  Jorge Lobo,et al.  Camera-Inertial Sensor modelling and alignment for Visual Navigation , 2003 .

[35]  Per-Erik Forssén,et al.  Efficient Video Rectification and Stabilisation for Cell-Phones , 2012, International Journal of Computer Vision.

[36]  A. B. Chatfield.  Fundamentals of high accuracy inertial navigation , 1997 .

[37]  Anastasios I. Mourikis,et al.  Estimator initialization in vision-aided inertial navigation with unknown camera-IMU calibration , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Standard SpeciÞcation Format Guide and Test Procedure for Single-Axis Interferometric Fiber Optic Gyros , 1998 .

[39]  S. Shankar Sastry,et al.  Geometric Models of Rolling-Shutter Cameras , 2005, ArXiv.

[40]  Yuanxin Wu,et al.  On 'A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation' , 2013, ArXiv.

[41]  Guy Le Besnerais,et al.  A Robust Indoor/Outdoor Navigation Filter Fusing Data from Vision and Magneto-Inertial Measurement Unit , 2017, Sensors.

[42]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[43]  Hugh F. Durrant-Whyte,et al.  Initial calibration and alignment of low-cost inertial navigation units for land vehicle applications , 1999, J. Field Robotics.

[44]  M.F. Golnaraghi,et al.  Initial calibration of an inertial measurement unit using an optical position tracking system , 2004, PLANS 2004. Position Location and Navigation Symposium (IEEE Cat. No.04CH37556).