Mapping, navigation, and learning for off-road traversal

The challenge in the DARPA Learning Applied to Ground Robots (LAGR) project is to autonomously navigate a small robot using stereo vision as the main sensor. During this project, we demonstrated a complete autonomous system for off-road navigation in unstructured environments, using stereo vision as the main sensor. The system is very robust—we can typically give it a goal position several hundred meters away and expect it to get there. In this paper we describe the main components that comprise the system, including stereo processing, obstacle and free space interpretation, long-range perception, online terrain traversability learning, visual odometry, map registration, planning, and control. At the end of 3 years, the system we developed outperformed all nine other teams in final blind tests over previously unseen terrain. © 2008 Wiley Periodicals, Inc.

[1]  Pietro Perona,et al.  Learning and prediction of slip from visual information , 2007, J. Field Robotics.

[2]  Kurt Konolige,et al.  FrameSLAM: From Bundle Adjustment to Real-Time Visual Mapping , 2008, IEEE Transactions on Robotics.

[3]  Andrew E. Johnson,et al.  Computer Vision on Mars , 2007, International Journal of Computer Vision.

[4]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[5]  Michael Happold,et al.  Enhancing Supervised Terrain Classification with Predictive Unsupervised Learning , 2006, Robotics: Science and Systems.

[6]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[7]  R. Bellman.  Dynamic programming. , 1957, Science.

[8]  Hans P. Moravec,et al.  High resolution maps from wide angle sonar , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[9]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[10]  Pietro Perona,et al.  Learning to predict slip for ground robots , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[11]  Anthony Stentz,et al.  Optimal and efficient path planning for partially-known environments , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[12]  Kurt Konolige,et al.  Visual Odometry Using Sparse Bundle Adjustment on an Autonomous Outdoor Vehicle , 2006, AMS.

[13]  James R. Bergen,et al.  Visual odometry for ground vehicle applications , 2006, J. Field Robotics.

[14]  Roberto Manduchi,et al.  Terrain perception for DEMO III , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[15]  Steven Dubowsky,et al.  Rapid physics-based rough-terrain rover planning with sensor and control uncertainty , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[16]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[17]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[18]  Kurt Konolige,et al.  A gradient method for realtime robot control , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[19]  Eduardo Mario Nebot,et al.  High accuracy navigation using laser range sensors in outdoor applications , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[20]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  Kurt Konolige,et al.  Real-time Localization in Outdoor Environments using Stereo Vision and Inexpensive GPS , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[22]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[23]  C. H. Chen,et al.  Handbook of Pattern Recognition and Computer Vision , 1993 .

[24]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[25]  Georgios Tziritas,et al.  Color and/or texture segmentation using deterministic relaxation and fast marching algorithms , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[26]  Roland Siegwart,et al.  An Interpolated Dynamic Navigation Function , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[27]  Kurt Konolige,et al.  Fast color/texture segmentation for outdoor robots , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[29]  M. Pietikäinen,et al.  TEXTURE ANALYSIS WITH LOCAL BINARY PATTERNS , 2004 .

[30]  Georgios Tziritas,et al.  Colour and texture segmentation using wavelet frame analysis, deterministic relaxation, and fast marching algorithms , 2004, J. Vis. Commun. Image Represent..

[31]  Alonzo Kelly,et al.  State Space Sampling of Feasible Motions for High Performance Mobile Robot Navigation in Highly Constrained Environments , 2007, FSR.

[32]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[33]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[34]  Clark F. Olson,et al.  Robust stereo ego-motion for long distance navigation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[35]  A. Rankin,et al.  Evaluation of stereo vision obstacle detection algorithms for off-road autonomous navigation , 2005 .

[36]  John J. Leonard,et al.  Consistent, Convergent, and Constant-Time SLAM , 2003, IJCAI.

[37]  Michel Dhome,et al.  Real Time Localization and 3D Reconstruction , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[38]  Kurt Konolige,et al.  Large-Scale Visual Odometry for Rough Terrain , 2007, ISRR.

[39]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[40]  Alonzo Kelly.  A Feedforward Control Approach to the Local Navigation Problem for Autonomous Vehicles , 1994 .

[41]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Larry H. Matthies,et al.  Robust and Efficient Stereo Feature Tracking for Visual Odometry , 2008, 2008 IEEE International Conference on Robotics and Automation.

[43]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[44]  Kurt Konolige,et al.  Incremental mapping of large cyclic environments , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[45]  Andrew Howard,et al.  Real-time stereo visual odometry for autonomous ground vehicles , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  Larry H. Matthies,et al.  Two years of Visual Odometry on the Mars Exploration Rovers , 2007, J. Field Robotics.

[47]  Sebastian Thrun,et al.  Large-Scale Robotic 3-D Mapping of Urban Structures , 2004, ISER.

[48]  J A Sethian,et al.  Computing geodesic paths on manifolds. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[50]  Ray Jarvis,et al.  3D Vision for Large-Scale Outdoor Environments , 2002 .