Autonomous flight in unstructured and unknown indoor environments

This paper presents our solution for enabling a quadrotor helicopter, equipped with a laser rangefinder sensor, to autonomously explore and map unstructured and unknown indoor environments. While these capabilities are already commodities on ground vehicles, air vehicles seeking the same performance face unique challenges. In this paper, we describe the difficulties in achieving fully autonomous helicopter flight, highlighting the differences between ground and helicopter robots that make it difficult to use algorithms developed for ground robots. We then describe our solutions to the key problems, including a multi-level sensing and control hierarchy, a high-speed laser scan-matching algorithm, EKF data fusion, and a high-level SLAM implementation. Finally, we show experimental results that illustrate the helicopter’s ability to navigate accurately and autonomously in unknown environments.

[1]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[2]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[3]  Jean-Claude Latombe,et al.  Sensory uncertainty field for mobile robot navigation , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[4]  H.F. Durrant-Whyte,et al.  A new approach for filtering nonlinear systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[5]  Jonathan Richard Shewchuk,et al.  Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator , 1996, WACG.

[6]  Brian Yamauchi,et al.  A frontier-based approach for autonomous exploration , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[7]  J. E. Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[8]  Gaurav S. Sukhatme,et al.  State estimation of an autonomous helicopter using Kalman filtering , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[9]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[10]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[11]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[13]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[14]  Sebastian Thrun,et al.  Learning occupancy grids with forward models , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[15]  H. Hirschmuller.  Improvements in real-time correlation-based stereo vision , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[16]  Heiko Hirschmüller,et al.  Improvements in real-time correlation-based stereo vision , 2001, CVPR 2001.

[17]  Wolfram Burgard,et al.  Robust Monte Carlo localization for mobile robots , 2001, Artif. Intell..

[18]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[19]  Petri Mähönen,et al.  TCP performance issues over wireless links , 2001, IEEE Commun. Mag..

[20]  Robert E. Mahony,et al.  Control of a quadrotor helicopter using visual feedback , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[21]  Trevor Darrell,et al.  Stereo tracking using ICP and normal flow constraint , 2002, Object recognition supported by user interaction for service robots.

[22]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[24]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[25]  Gaurav S. Sukhatme,et al.  A tale of two helicopters , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[26]  Vincent Roca,et al.  Analysis of FEC Codes for Partially Reliable Media Broadcasting Schemes , 2004, MIPS.

[27]  Sebastian Thrun,et al.  6D SLAM with an application in autonomous mine mapping , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[28]  C. Taylor,et al.  Vision-based pose estimation and control of a model helicopter , 2004, Proceedings of the IEEE International Conference on Mechatronics, 2004. ICM '04..

[29]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[30]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[31]  Carl-Fredrik Westin,et al.  Robust Generalized Total Least Squares Iterative Closest Point Registration , 2004, MICCAI.

[32]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[33]  Joachim Hertzberg,et al.  High-speed laser localization for mobile robots , 2005, Robotics Auton. Syst..

[34]  Sai-Ming Li,et al.  Forest fire monitoring with multiple small UAVs , 2005, Proceedings of the 2005, American Control Conference, 2005..

[35]  Hiroshi Tokutake,et al.  Navigation of Small and Lightweight Helicopter , 2005 .

[36]  Dirk Hähnel,et al.  Mapping with mobile robots , 2005 .

[37]  Roland Siegwart,et al.  Towards Autonomous Indoor Micro VTOL , 2005, Auton. Robots.

[38]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[39]  Sebastian Thrun,et al.  Discriminative Training of Kalman Filters , 2005, Robotics: Science and Systems.

[40]  Takeo Kanade,et al.  Online dense local 3D world reconstruction from stereo image sequences , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[42]  Fatih Murat Porikli,et al.  Achieving real-time object detection and tracking under extreme conditions , 2006, Journal of Real-Time Image Processing.

[43]  Gaurav S. Sukhatme,et al.  Visual Servoing for Tracking Features in Urban Areas Using an Autonomous Helicopter , 2006 .

[44]  Paul Newman,et al.  Using laser range data for 3D SLAM in outdoor environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[45]  Mario J. Valenti,et al.  Estimation and Control of a Quadrotor Vehicle Using Monocular Vision and Moire Patterns , 2006 .

[46]  Hugh F. Durrant-Whyte,et al.  Recursive Bayesian search-and-tracking using coordinated uavs for lost targets , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[47]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[48]  Sudipta N. Sinha,et al.  REAL-TIME VIDEO-BASED RECONSTRUCTION OF URBAN ENVIRONMENTS , 2007 .

[49]  Wolfram Burgard,et al.  Learning maps in 3D using attitude and noisy vision sensors , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[50]  Gerd Hirzinger,et al.  Energy-efficient Autonomous Four-rotor Flying Robot Controlled at 1 kHz , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[51]  Ashutosh Saxena,et al.  Make3D: Learning 3D Scene Structure from a Single Still Image , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Shai Avidan.  Ensemble Tracking , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Sebastian Thrun,et al.  Autonomous Helicopter Tracking and Localization Using a Self-surveying Camera Array , 2007, Int. J. Robotics Res..

[55]  S. Shankar Sastry,et al.  Autonomous Vision-based Landing and Terrain Mapping Using an MPC-controlled Unmanned Rotorcraft , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[56]  Gaurav S. Sukhatme,et al.  Landing a Helicopter on a Moving Target , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[57]  Dario Floreano,et al.  Quadrotor Using Minimal Sensing For Autonomous Indoor Flight , 2007 .

[58]  Kimon P. Valavanis,et al.  Control Fundamentals of Small / Miniature Helicopters - A Survey , 2007 .

[59]  Claire J. Tomlin,et al.  Quadrotor Helicopter Flight Dynamics and Control: Theory and Experiment , 2007 .

[60]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[61]  R. Rifkin,et al.  Notes on Regularized Least Squares , 2007 .

[62]  Sebastian Scherer,et al.  Flying Fast and Low Among Obstacles , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[63]  Alborz Geramifard,et al.  Co-ordinated Tracking and Planning Using Air and Ground Vehicles , 2008, ISER.

[64]  Andrew Howard,et al.  Real-time stereo visual odometry for autonomous ground vehicles , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  Edwin Olson,et al.  Robust and efficient robotic mapping , 2008 .

[66]  J. Karl Hedrick,et al.  A multiple UAV system for vision-based search and localization , 2008, 2008 American Control Conference.

[67]  Neil G. Johnson,et al.  Vision-Assisted Control of a Hovering Air Vehicle in an Indoor Setting , 2008 .

[68]  Lina María Paz,et al.  Large-Scale 6-DOF SLAM With Stereo-in-Hand , 2008, IEEE Transactions on Robotics.

[69]  Nicholas Roy,et al.  Planning in information space for a quadrotor helicopter in a GPS-denied environment , 2008, 2008 IEEE International Conference on Robotics and Automation.

[70]  Luke Fletcher,et al.  A perception-driven autonomous urban vehicle , 2008 .

[71]  B. Bethke,et al.  Real-time indoor autonomous vehicle test environment , 2008, IEEE Control Systems.

[72]  Spencer Greg Ahrens.  Vision-based guidance and control of a hovering vehicle in unknown environments , 2008 .

[73]  Paul Newman,et al.  Accelerated appearance-only SLAM , 2008, 2008 IEEE International Conference on Robotics and Automation.

[74]  Luca Iocchi,et al.  Autonomous Indoor Hovering with a Quadrotor , 2008 .

[75]  David W. Murray,et al.  Improving the Agility of Keyframe-Based SLAM , 2008, ECCV.

[76]  Soon-Jo Chung,et al.  Mono-vision corner SLAM for indoor navigation , 2008, 2008 IEEE International Conference on Electro/Information Technology.

[77]  ZuWhan Kim.  Real time object tracking based on dynamic feature grouping with background subtraction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[78]  Pieter Abbeel,et al.  Learning for control from multiple demonstrations , 2008, ICML '08.

[79]  Wolfram Burgard,et al.  Autonomous Indoors Navigation using a Small-Size Quadrotor , 2008 .

[80]  Edwin Olson,et al.  Lightweight Communications and Marshalling for Low-Latency Interprocess Communication , 2009 .

[81]  Huiyu Zhou,et al.  Object tracking using SIFT features and mean shift , 2009, Comput. Vis. Image Underst..

[82]  Wolfram Burgard,et al.  Towards a navigation system for autonomous indoor flying , 2009, 2009 IEEE International Conference on Robotics and Automation.

[83]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[84]  Gaurav S. Sukhatme,et al.  Vision‐based navigation through urban canyons , 2009, J. Field Robotics.

[85]  Edwin Olson,et al.  Real-time correlative scan matching , 2009, 2009 IEEE International Conference on Robotics and Automation.

[86]  Edwin Olson,et al.  LCM: Lightweight Communications and Marshalling , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.