Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors

The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To the best of our knowledge, this is the first example of extending the use of these smart cameras from hovering-like motions to odometry estimation, producing estimates that are usable during flight times of several minutes. In order to validate and defend the simplest algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside with a large number of algorithm modifications defended in earlier literature on visual-inertial odometry, showing that their impact on filter performance is minimal. To close the control loop, a non-linear controller operating in the special Euclidean group SE(3) is able to drive, based on the estimated vehicle’s state, a quadrotor platform in 3D space guaranteeing the asymptotic stability of 3D position and heading. All the estimation and control tasks are solved on board and in real time on a limited computational unit. The proposed approach is validated through simulations and experimental results, which include comparisons with ground-truth data provided by a motion capture system. For the benefit of the community, we make the source code public.

[1]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[3]  Vijay Kumar,et al.  Cooperative localization and mapping of MAVs using RGB-D sensors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Teresa A. Vidal-Calleja,et al.  Impact of Landmark Parametrization on Monocular EKF-SLAM with Points and Lines , 2011, International Journal of Computer Vision.

[5]  Vijay Kumar,et al.  Vision-Based State Estimation and Trajectory Control Towards High-Speed Flight with a Quadrotor , 2013, Robotics: Science and Systems.

[6]  Vijay Kumar,et al.  Toward image based visual servoing for aerial grasping and perching , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Marc Pollefeys,et al.  Vision-based autonomous mapping and exploration using a quadrotor MAV , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Taeyoung Lee,et al.  Nonlinear Robust Tracking Control of a Quadrotor UAV on SE(3) , 2013 .

[9]  Mingyang Li,et al.  Improving the accuracy of EKF-based visual-inertial odometry , 2012, 2012 IEEE International Conference on Robotics and Automation.

[10]  Marc Pollefeys,et al.  An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications , 2013, 2013 IEEE International Conference on Robotics and Automation.

[11]  Stefano Soatto,et al.  Visual-inertial navigation, mapping and localization: A scalable real-time causal approach , 2011, Int. J. Robotics Res..

[12]  Roland Siegwart,et al.  Monocular‐SLAM–based navigation for autonomous micro helicopters in GPS‐denied environments , 2011, J. Field Robotics.

[13]  Nicolas Mansard,et al.  RT-SLAM: A Generic and Real-Time Visual SLAM Implementation , 2011, ICVS.

[14]  Marc Pollefeys,et al.  PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision , 2012, Auton. Robots.

[15]  Agostino Martinelli,et al.  Vision and IMU Data Fusion: Closed-Form Solutions for Attitude, Speed, Absolute Scale, and Bias Determination , 2012, IEEE Transactions on Robotics.

[16]  Roland Siegwart,et al.  A synchronized visual-inertial sensor system with FPGA pre-processing for accurate real-time SLAM , 2014, ICRA 2014.

[17]  M. Fliess,et al.  Flatness and defect of non-linear systems: introductory theory and examples , 1995 .

[18]  Paolo Rocco,et al.  Trajectory Generation for Unmanned Aerial Manipulators Through Quadratic Programming , 2017, IEEE Robotics and Automation Letters.

[19]  Kazuya Yoshida,et al.  Collaborative mapping of an earthquake‐damaged building via ground and aerial robots , 2012, J. Field Robotics.

[20]  Lorenzo Marconi,et al.  Impedance control of an aerial manipulator , 2012, 2012 American Control Conference (ACC).

[21]  Juan Andrade-Cetto,et al.  High-frequency MAV state estimation using low-cost inertial and optical flow measurement units , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Hannes Sommer,et al.  Fusion of optical flow and inertial measurements for robust egomotion estimation , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Vijay Kumar,et al.  Inspection of Penstocks and Featureless Tunnel-like Environments Using Micro UAVs , 2013, FSR.

[24]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[25]  Vijay Kumar,et al.  Minimum snap trajectory generation and control for quadrotors , 2011, 2011 IEEE International Conference on Robotics and Automation.

[26]  Sammy Omari,et al.  Metric visual-inertial navigation system using single optical flow feature , 2013, 2013 European Control Conference (ECC).

[27]  Vincenzo Lippiello,et al.  Task priority control for aerial manipulation , 2014, 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics (2014).

[28]  J. S. Ortega Quaternion kinematics for the error-state KF , 2016 .

[29]  A. D. Lewis,et al.  Geometric Control of Mechanical Systems , 2004, IEEE Transactions on Automatic Control.

[30]  Roland Siegwart,et al.  Vision based MAV navigation in unknown and unstructured environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[31]  Vincenzo Lippiello,et al.  Uncalibrated Visual Servo for Unmanned Aerial Manipulation , 2017, IEEE/ASME Transactions on Mechatronics.

[32]  Anup Goyal,et al.  Extended Kalman Filter vs. Error State Kalman Filter for Aircraft Attitude Estimation , 2011 .

[33]  Robert E. Mahony,et al.  Landing a VTOL Unmanned Aerial Vehicle on a Moving Platform Using Optical Flow , 2012, IEEE Transactions on Robotics.

[34]  Darius Burschka,et al.  Toward a Fully Autonomous UAV: Research Platform for Indoor and Outdoor Urban Search and Rescue , 2012, IEEE Robotics & Automation Magazine.

[35]  Vijay Kumar,et al.  Smartphones power flying robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[36]  Agostino Martinelli,et al.  Visual-inertial structure from motion: Observability and resolvability , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Flavio Fontana,et al.  Autonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle , 2016, J. Field Robotics.

[38]  Vijay Kumar,et al.  The GRASP Multiple Micro-UAV Testbed , 2010, IEEE Robotics & Automation Magazine.

[39]  Vijay Kumar,et al.  Autonomous indoor 3D exploration with a micro-aerial vehicle , 2012, 2012 IEEE International Conference on Robotics and Automation.

[40]  Roland Siegwart,et al.  Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments , 2012, 2012 IEEE International Conference on Robotics and Automation.

[41]  Anastasios I. Mourikis,et al.  High-precision, consistent EKF-based visual-inertial odometry , 2013, Int. J. Robotics Res..

[42]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[43]  A. Masi,et al.  NEW INFRARED TIME OF-FLIGHT MEASUREMENT SENSOR FOR ROBOTIC PLATFORMS , 2014 .

[44]  Dimitrios G. Kottas,et al.  Camera-IMU-based localization: Observability analysis and consistency improvement , 2014, Int. J. Robotics Res..

[45]  N. Trawny,et al.  Indirect Kalman Filter for 3 D Attitude Estimation , 2005 .

[46]  Stergios I. Roumeliotis,et al.  Augmenting inertial navigation with image-based motion estimation , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).