Stereo-based all-terrain obstacle detection using visual saliency

This paper proposes a hybridization of two well-known stereo-based obstacle detection techniques for all-terrain environments. While one of the techniques is employed for the detection of large obstacles, the other is used for the detection of small ones. This combination of techniques opportunistically exploits their complementary properties to reduce computation and improve detection accuracy. Being particularly computation intensive and prone to generate a high false-positive rate in the face of noisy three-dimensional point clouds, the technique for small obstacle detection is further extended in two directions. The goal of the first extension is to reduce both problems by focusing the detection on those regions of the visual field that detach more from the background and, consequently, are more likely to contain an obstacle. This is attained by means of spatially varying the data density of the input images according to their visual saliency. The second extension refers to the use of a novel voting mechanism, which further improves robustness. Extensive experimental results confirm the ability of the proposed method to robustly detect obstacles up to a range of 20 m on uneven terrain. Moreover, the model runs at 5 Hz on 640 × 480 stereo images. © 2010 Wiley Periodicals, Inc. © 2011 Wiley Periodicals, Inc.

[1]  Alonzo Kelly,et al.  Rough Terrain Autonomous Mobility—Part 2: An Active Vision, Predictive Control Approach , 1998, Auton. Robots.

[2]  Donald B. Gennery,et al.  Traversability Analysis and Path Planning for a Planetary Rover , 1999, Auton. Robots.

[3]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[4]  George K. I. Mann,et al.  A task-driven object-based attention model for robots , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[5]  Cang Ye,et al.  Navigating a Mobile Robot by a Traversability Field Histogram , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Marc M. Van Hulle,et al.  Ground plane estimation based on dense stereo disparity , 2008 .

[7]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[8]  Stefan Schaal,et al.  Overt visual attention for a humanoid robot , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[9]  Simon Lacroix,et al.  Autonomous Rover Navigation on Unknown Terrains Functions and Integration , 2000, ISER.

[10]  Simone Frintrop,et al.  Goal-Directed Search with a Top-Down Modulated Computational Attention System , 2005, DAGM-Symposium.

[11]  Martial Hebert,et al.  Natural terrain classification using three‐dimensional ladar data for ground robot mobility , 2006, J. Field Robotics.

[12]  J. C. van den Heuvel,et al.  Stereo based Obstacle Detection with Uncertainty in Rough Terrain , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[13]  Yann LeCun,et al.  Learning long‐range vision for autonomous off‐road driving , 2009, J. Field Robotics.

[14]  Ole Ravn,et al.  Vision Assisted Laser Scanner Navigation for Autonomous Robots , 2006, ISER.

[15]  Homayoun Seraji Traversability index: a new concept for planetary rovers , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[16]  Paolo Grisleri,et al.  Off-Road Path and Obstacle Detection Using Decision Networks and Stereo Vision , 2007, IEEE Transactions on Intelligent Transportation Systems.

[17]  Giulio Sandini,et al.  Object-based Visual Attention: a Model for a Behaving Robot , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[18]  Karsten Berns,et al.  Stereo-Vision-Based Obstacle Avoidance in Rough Outdoor Terrain , 2005 .

[19]  Roberto Manduchi,et al.  Fast and reliable obstacle detection and segmentation for cross-country navigation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[20]  Wolfram Burgard,et al.  Learning predictive terrain models for legged robot locomotion , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Marilyn N. Abrams,et al.  Concealment and Obstacle Detection for Autonomous Driving , 1999 .

[22]  Alex D. Hwang,et al.  A model of top-down attentional control during visual search in complex scenes. , 2009, Journal of vision.

[23]  Sanjiv Singh,et al.  Obstacle detection in smooth high curvature terrain , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[24]  Andreas Birk,et al.  Hough based terrain classification for realtime detection of drivable ground , 2008, J. Field Robotics.

[25]  Larry Matthies,et al.  Stereo vision and rover navigation software for planetary exploration , 2002, Proceedings, IEEE Aerospace Conference.

[26]  Paulo Santos,et al.  Cross-country obstacle detection: Space-variant resolution and outliers removal , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Nicholas J. Butko,et al.  Active perception , 2010 .

[28]  Larry H. Matthies,et al.  Foliage discrimination using a rotating ladar , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[29]  Roberto Manduchi,et al.  Terrain perception for DEMO III , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[30]  Homayoun Seraji Safety measures for terrain classification and safest site selection , 2006, Auton. Robots.

[31]  Ben Taskar,et al.  Online, self-supervised terrain classification via discriminatively trained submodular Markov random fields , 2008, 2008 IEEE International Conference on Robotics and Automation.

[32]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[33]  Alberto Broggi,et al.  Obstacle Detection with Stereo Vision for Off-Road Vehicle Navigation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[34]  Larry H. Matthies,et al.  Perception control for obstacle detection by a cross-country rover , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[35]  William Whittaker,et al.  A robust approach to high‐speed navigation for unrehearsed desert terrain , 2006, J. Field Robotics.

[36]  Roberto Manduchi,et al.  Obstacle Detection and Terrain Classification for Autonomous Off-Road Navigation , 2005, Auton. Robots.

[37]  Anthony Stentz,et al.  Online adaptive rough-terrain navigation vegetation , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[38]  Andrew Y. Ng,et al.  Stereo vision and terrain modeling for quadruped robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[39]  José Barata,et al.  A saliency-based solution for robust off-road obstacle detection , 2010, 2010 IEEE International Conference on Robotics and Automation.

[40]  J.J. Biesiadecki,et al.  The Mars Exploration Rover surface mobility flight software driving ambition , 2006, 2006 IEEE Aerospace Conference.

[41]  Sebastian Thrun,et al.  Probabilistic Terrain Analysis For High-Speed Desert Driving , 2006, Robotics: Science and Systems.

[42]  Christian Hoffmann,et al.  Fast Object Hypotheses Generation Using 3D Position and 3D Motion , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[43]  Simone Frintrop,et al.  VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search , 2006, Lecture Notes in Computer Science.

[44]  Simon Lacroix,et al.  Autonomous Rover Navigation on Unknown Terrains: Functions and Integration , 2000, Int. J. Robotics Res..

[45]  Robert C. Bolles,et al.  Mapping, navigation, and learning for off‐road traversal , 2009, J. Field Robotics.

[46]  José Barata,et al.  Saliency-Based Obstacle Detection and Ground-Plane Estimation for Off-Road Vehicles , 2009, ICVS.

[47]  James J. Little,et al.  Curious George: An attentive semantic robot , 2008, Robotics Auton. Syst..

[48]  Robert E. Mahony,et al.  Spatio-Temporal RANSAC for Robust Estimation of Ground Plane in Video Range Images for Automotive Applications , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[49]  Sanjiv Singh,et al.  Obstacle detection using adaptive color segmentation and color stereo homography , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[50]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[51]  N. Hautiere,et al.  Road Segmentation Supervised by an Extended V-Disparity Algorithm for Autonomous Navigation , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[52]  Larry H. Matthies,et al.  Learning long-range terrain classification for autonomous navigation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[53]  Michael Beetz,et al.  Leaving Flatland: Efficient real‐time three‐dimensional perception and motion planning , 2009, J. Field Robotics.

[54]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[55]  Tommy Chang,et al.  Fusing Ladar and Color Image Information for Mobile Robot Feature Detection and Tracking , 2002 .

[56]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[57]  Reid G. Simmons,et al.  Recent progress in local and global traversability for planetary rovers , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[58]  Henrik I. Christensen,et al.  Simultaneous Robot Localization and Mapping Based on a Visual Attention System , 2008, WAPCV.

[59]  Paul Newman,et al.  SLAM-Loop Closing with Visually Salient Features , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[60]  Frans C. A. Groen,et al.  Obstacle detection during day and night conditions using stereo vision , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[61]  Gordon Cheng,et al.  Biologically Based Top-Down Attention Modulation for Humanoid Interactions , 2008, Int. J. Humanoid Robotics.

[62]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.