Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles

In this paper, we present a visual simultaneous localization and mapping (SLAM) system which integrates measurements from multiple cameras to achieve robust pose tracking for autonomous navigation of micro aerial vehicles (MAVs) in unknown complex environments. We analyze the iterative optimizations for pose tracking and map refinement of visual SLAM in multi-camera cases. The analysis ensures the soundness and accuracy of each optimization update. A well-known monocular visual SLAM system is extended to utilize two cameras with non-overlapping fields of view (FOVs) in the final implementation. The resulting visual SLAM system enables autonomous navigation of an MAV in complex scenarios. The theory behind this system can easily be extended to multi-camera configurations, when the onboard computational capability allows this. For operations in large-scale environments, we modify the resulting visual SLAM system to be a constant-time robust visual odometry. To form a full visual SLAM system, we further implement an efficient back-end for loop closing. The back-end maintains a keyframe-based global map, which is also used for loop-closure detection. An adaptive-window pose-graph optimization method is proposed to refine keyframe poses of the global map and thus correct pose drift that is inherent in the visual odometry. We demonstrate the efficiency of the proposed visual SLAM system for applications onboard of MAVs in experiments with both autonomous and manual flights. The pose tracking results are compared with ground truth data provided by an external tracking system. A SLAM system integrating measurements from multiple cameras for MAVs is proposed.No overlap in the respective fields of view of the multiple cameras is required.Robust pose-tracking can be achieved in complex environments.Mathematical analysis on the iterative optimizations in visual SLAM is provided.The efficiency of the proposed visual SLAM system is demonstrated onboard of MAVs.

[1]  Hauke Strasdat,et al.  Visual SLAM: Why filter? , 2012, Image Vis. Comput..

[2]  Andreas Zell,et al.  On-board dual-stereo-vision for autonomous quadrotor navigation , 2013, 2013 International Conference on Unmanned Aircraft Systems (ICUAS).

[3]  Andreas Zell,et al.  Robust Onboard Visual SLAM for Autonomous MAVs , 2014, IAS.

[4]  Roland Siegwart,et al.  Monocular Vision for Long‐term Micro Aerial Vehicle State Estimation: A Compendium , 2013, J. Field Robotics.

[5]  Frank Dellaert,et al.  Visual SLAM with a Multi-Camera Rig , 2006 .

[6]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[7]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[8]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[9]  Jose Luis Blanco,et al.  A tutorial on SE(3) transformation parameterizations and on-manifold optimization , 2012 .

[10]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[11]  Andreas Zell,et al.  A robust nonlinear controller for nontrivial quadrotor maneuvers: Approach and verification , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Andreas Zell,et al.  Using depth in visual simultaneous localisation and mapping , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Marc Pollefeys,et al.  Infrastructure-based calibration of a multi-camera rig , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Roland Siegwart,et al.  Real-time metric state estimation for modular vision-inertial systems , 2011, 2011 IEEE International Conference on Robotics and Automation.

[15]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[16]  Hauke Strasdat,et al.  Real-time monocular SLAM: Why filter? , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[18]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Andreas Zell,et al.  Efficient onbard RGBD-SLAM for autonomous MAVs , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Marc Pollefeys,et al.  PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision , 2012, Auton. Robots.

[21]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[22]  Vijay Kumar,et al.  Trajectory Generation and Control for Precise Aggressive Maneuvers with Quadrotors , 2010, ISER.

[23]  Adam Harmat,et al.  Parallel Tracking and Mapping with Multiple Cameras on an Unmanned Aerial Vehicle , 2012, ICIRA.

[24]  Quan Pan,et al.  Pose estimation for multi-camera systems , 2017, 2017 IEEE International Conference on Unmanned Systems (ICUS).

[25]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Marc Pollefeys,et al.  Motion Estimation for Self-Driving Cars with a Generalized Camera , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[28]  Roland Siegwart,et al.  A robust and modular multi-sensor fusion approach applied to MAV navigation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Robert Pless,et al.  Using many cameras as one , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Zhiqiang Zheng,et al.  A robust omnidirectional vision sensor for soccer robots , 2011 .

[31]  Vijay Kumar,et al.  Vision-Based State Estimation and Trajectory Control Towards High-Speed Flight with a Quadrotor , 2013, Robotics: Science and Systems.

[32]  Marc Pollefeys,et al.  Structureless pose-graph loop-closure with a multi-camera system on a self-driving car , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Andreas Zell,et al.  Autonomous Landing of MAVs on an Arbitrarily Textured Landing Site Using Onboard Monocular Vision , 2014, J. Intell. Robotic Syst..

[34]  Andreas Zell,et al.  DCTAM: Drift-corrected tracking and mapping for autonomous micro aerial vehicles , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[35]  Roland Siegwart,et al.  Onboard IMU and monocular vision based control for MAVs in unknown in- and outdoor environments , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Roland Siegwart,et al.  Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments , 2012, 2012 IEEE International Conference on Robotics and Automation.

[37]  Anastasios I. Mourikis,et al.  High-precision, consistent EKF-based visual-inertial odometry , 2013, Int. J. Robotics Res..

[38]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[39]  Kazuya Yoshida,et al.  Collaborative mapping of an earthquake‐damaged building via ground and aerial robots , 2012, J. Field Robotics.

[40]  Teresa A. Vidal-Calleja,et al.  Fusing Monocular Information in Multicamera SLAM , 2008, IEEE Transactions on Robotics.

[41]  Andreas Zell,et al.  Visual SLAM for autonomous MAVs with dual cameras , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[42]  Dimitrios G. Kottas,et al.  Camera-IMU-based localization: Observability analysis and consistency improvement , 2014, Int. J. Robotics Res..

[43]  Kurt Konolige,et al.  Double window optimisation for constant time visual SLAM , 2011, 2011 International Conference on Computer Vision.

[44]  Frank Dellaert,et al.  Probabilistic structure matching for visual SLAM with a multi-camera rig , 2010, Comput. Vis. Image Underst..

[45]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[46]  Vijay Kumar,et al.  Vision-based state estimation for autonomous rotorcraft MAVs in complex environments , 2013, 2013 IEEE International Conference on Robotics and Automation.

[47]  Tom Drummond,et al.  Monocular SLAM as a Graph of Coalesced Observations , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[48]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[49]  Andreas Zell,et al.  Robust and efficient volumetric occupancy mapping with an application to stereo vision , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[50]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[51]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[52]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[53]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[54]  Dorian Gálvez-López,et al.  Bags of Binary Words for Fast Place Recognition in Image Sequences , 2012, IEEE Transactions on Robotics.

[55]  Andreas Zell,et al.  An Onboard Monocular Vision System for Autonomous Takeoff, Hovering and Landing of a Micro Aerial Vehicle , 2012, Journal of Intelligent & Robotic Systems.

[56]  Gamini Dissanayake,et al.  Linear MonoSLAM: A linear approach to large-scale monocular SLAM problems , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[57]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[58]  Jizhong Xiao,et al.  Autonomous quadrotor flight using onboard RGB-D visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[59]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[60]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[61]  Marc Pollefeys,et al.  Self-calibration and visual SLAM with a multi-camera system on a micro aerial vehicle , 2014, Auton. Robots.

[62]  Marc Pollefeys,et al.  Vision-based autonomous mapping and exploration using a quadrotor MAV , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[63]  Adam Harmat,et al.  Multi-camera parallel tracking and mapping with non-overlapping fields of view , 2015, Int. J. Robotics Res..

[64]  Andrew J. Davison,et al.  SLAM-based automatic extrinsic calibration of a multi-camera rig , 2011, 2011 IEEE International Conference on Robotics and Automation.