Observability Analysis and Performance Evaluation of EKF-Based Visual-Inertial Odometry With Online Intrinsic Camera Parameter Calibration

In this paper, we focus on the problem of online intrinsic camera parameter calibration for a visual-inertial system. Imprecise intrinsic camera parameters will result in unreliable pose estimation or even cause estimator divergence. Specifically, we present a nonlinear observability analysis of the system and prove that there are four unobservable directions spanning the right nullspace of the observability matrix, i.e., the rotation about the gravity vector and the positions in the global frame. We propose an extended Kalman filter-based visual-inertial odometry method for calibrating intrinsic camera parameters while estimating the pose simultaneously. The observability properties and the performance of the estimator are validated using both the simulated and real-world datasets.

[1]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[2]  Frank Dellaert,et al.  On-Manifold Preintegration for Real-Time Visual--Inertial Odometry , 2015, IEEE Transactions on Robotics.

[3]  N. Trawny,et al.  Indirect Kalman Filter for 3 D Attitude Estimation , 2005 .

[4]  Michael Bosse,et al.  Keyframe-based visual–inertial odometry using nonlinear optimization , 2015, Int. J. Robotics Res..

[5]  Stergios I. Roumeliotis,et al.  A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation , 2008, IEEE Transactions on Robotics.

[6]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[7]  Juan D. Tardós,et al.  Visual-Inertial Monocular SLAM With Map Reuse , 2016, IEEE Robotics and Automation Letters.

[8]  Emanuele Menegatti,et al.  A robust and easy to implement method for IMU calibration without external equipments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[9]  John J. Leonard,et al.  Towards consistent visual-inertial navigation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  Anastasios I. Mourikis,et al.  Online temporal calibration for camera–IMU systems: Theory and algorithms , 2014, Int. J. Robotics Res..

[12]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[13]  Roland Siegwart,et al.  Robust visual inertial odometry using a direct EKF-based approach , 2015, IROS 2015.

[14]  Dimitrios G. Kottas,et al.  Camera-IMU-based localization: Observability analysis and consistency improvement , 2014, Int. J. Robotics Res..

[15]  A. Krener,et al.  Nonlinear controllability and observability , 1977 .

[16]  Roland Siegwart,et al.  The EuRoC micro aerial vehicle datasets , 2016, Int. J. Robotics Res..

[17]  Daniel Cremers,et al.  Direct Sparse Visual-Inertial Odometry Using Dynamic Marginalization , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Dimitrios G. Kottas,et al.  Efficient Visual-Inertial Navigation using a Rolling-Shutter Camera with Inaccurate Timestamps , 2014, Robotics: Science and Systems.

[19]  Roland Siegwart,et al.  Unified temporal and spatial calibration for multi-sensor systems , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  M. Shuster.  A survey of attitude representation , 1993 .

[21]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[22]  Anastasios I. Mourikis,et al.  High-fidelity sensor modeling and self-calibration in vision-aided inertial navigation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Stergios I. Roumeliotis,et al.  Stochastic cloning: a generalized framework for processing relative state measurements , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[24]  Stergios I. Roumeliotis,et al.  A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[25]  Anastasios I. Mourikis,et al.  High-precision, consistent EKF-based visual-inertial odometry , 2013, Int. J. Robotics Res..

[26]  Shaojie Shen,et al.  VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator , 2017, IEEE Transactions on Robotics.

[27]  Javier Civera,et al.  Camera self-calibration for sequential Bayesian structure from motion , 2009, 2009 IEEE International Conference on Robotics and Automation.