Accurate and Robust Monocular SLAM with Omnidirectional Cameras

Simultaneous localization and mapping (SLAM) are fundamental elements for many emerging technologies, such as autonomous driving and augmented reality. For this paper, to get more information, we developed an improved monocular visual SLAM system by using omnidirectional cameras. Our method extends the ORB-SLAM framework with the enhanced unified camera model as a projection function, which can be applied to catadioptric systems and wide-angle fisheye cameras with 195 degrees field-of-view. The proposed system can use the full area of the images even with strong distortion. For omnidirectional cameras, a map initialization method is proposed. We analytically derive the Jacobian matrices of the reprojection errors with respect to the camera pose and 3D position of points. The proposed SLAM has been extensively tested in real-world datasets. The results show positioning error is less than 0.1% in a small indoor environment and is less than 1.5% in a large environment. The results demonstrate that our method is real-time, and increases its accuracy and robustness over the normal systems based on the pinhole model.

[1]  Xiaowei Wang,et al.  Spherical-Model-Based SLAM on Full-View Images for Indoor Environments , 2018, Applied Sciences.

[2]  Daniel Cremers,et al.  The Double Sphere Camera Model , 2018, 2018 International Conference on 3D Vision (3DV).

[3]  Juho Kannala,et al.  A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[5]  Jörg Stückler,et al.  The TUM VI Benchmark for Evaluating Visual-Inertial Odometry , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Torsten Sattler,et al.  Direct visual odometry for a fisheye-stereo camera , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[8]  Xinyu Zhang,et al.  Real-time Omnidirectional Visual SLAM with Semi-Dense Mapping , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[9]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jörg Stückler,et al.  Omnidirectional DSO: Direct Sparse Odometry With Fisheye Cameras , 2018, IEEE Robotics and Automation Letters.

[11]  Francesc Moreno-Noguer,et al.  PL-SLAM: Real-time monocular visual SLAM with points and lines , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[12]  David Valiente,et al.  Improved Omnidirectional Odometry for a View-Based Mapping Approach , 2017, Sensors.

[13]  Luis Payá,et al.  Robust Visual Localization with Dynamic Uncertainty Management in Omnidirectional SLAM , 2017 .

[14]  Kostas Daniilidis,et al.  A Unifying Theory for Central Panoramic Systems and Practical Applications , 2000, ECCV.

[15]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[16]  Roland Siegwart,et al.  Unified temporal and spatial calibration for multi-sensor systems , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Yongping Zhai,et al.  RGB-D SLAM Using Point–Plane Constraints for Indoor Environments † , 2019, Sensors.

[18]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[19]  Changjae Kim,et al.  Analysis of Fish-Eye Lens Camera Self-Calibration , 2019, Sensors.

[20]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[21]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[22]  Daniel Cremers,et al.  Large-scale direct SLAM for omnidirectional cameras , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[24]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[25]  Philippe Martinet,et al.  An Enhanced Unified Camera Model , 2016, IEEE Robotics and Automation Letters.

[26]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[27]  Lionel Heng,et al.  Semi-direct visual odometry for a fisheye-stereo camera , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Olivier Faugeras,et al.  Motion and Structure from Motion in a piecewise Planar Environment , 1988, Int. J. Pattern Recognit. Artif. Intell..