Finding multiple lanes in urban road networks with vision and lidar

This paper describes a system for detecting and estimating the properties of multiple travel lanes in an urban road network from calibrated video imagery and laser range data acquired by a moving vehicle. The system operates in real-time in several stages on multiple processors, fusing detected road markings, obstacles, and curbs into a stable non-parametric estimate of nearby travel lanes. The system incorporates elements of a provided piecewise-linear road network as a weak prior.Our method is notable in several respects: it detects and estimates multiple travel lanes; it fuses asynchronous, heterogeneous sensor streams; it handles high-curvature roads; and it makes no assumption about the position or orientation of the vehicle with respect to the road.We analyze the system’s performance in the context of the 2007 DARPA Urban Challenge. With five cameras and thirteen lidars, our method was incorporated into a closed-loop controller to successfully guide an autonomous vehicle through a 90 km urban course at speeds up to 40 km/h amidst moving traffic.

[1]  Luke Fletcher,et al.  Context Sensitive Driver Assistance Based on Gaze - Road Scene Correlation , 2006, ISER.

[2]  Shumeet Baluja,et al.  Evolution of an artificial neural network based autonomous land vehicle controller , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[3]  Dong Joong Kang,et al.  Road lane segmentation using dynamic programming for active safety vehicles , 2003, Pattern Recognit. Lett..

[4]  Massimo Bertozzi,et al.  Vision-based intelligent vehicles: State of the art and perspectives , 2000, Robotics Auton. Syst..

[5]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[6]  John Haines.  To Turn Back , 1964 .

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Wei-Bin Zhang.  A roadway information system for vehicle guidance/control , 1991, Vehicle Navigation and Information Systems Conference, 1991.

[9]  Luke Fletcher,et al.  Simultaneous local and global state estimation for robotic navigation , 2009, 2009 IEEE International Conference on Robotics and Automation.

[10]  Todd Jochem,et al.  Rapidly Adapting Machine Vision for Automated Vehicle Steering , 1996, IEEE Expert.

[11]  Mohan M. Trivedi,et al.  Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation , 2006, IEEE Transactions on Intelligent Transportation Systems.

[12]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[13]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[14]  Takeo Kanade,et al.  Vision and Navigation for the Carnegie-Mellon Navlab , 1987 .

[15]  Joon Woong Lee,et al.  A lane-departure identification based on LBPE, Hough transform, and linear regression , 2005, Comput. Vis. Image Underst..

[16]  Ted R Miller.  BENEFIT-COST ANALYSIS OF LANE MARKING , 1992 .

[17]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[18]  Luke Fletcher,et al.  A perception‐driven autonomous urban vehicle , 2008, J. Field Robotics.

[19]  Ernst D. Dickmanns,et al.  Recursive 3-D Road and Relative Ego-State Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Alexander Zelinsky,et al.  Vision In and Out of Vehicles: Integrated Driver and Road Scene Monitoring , 2002, ISER.

[21]  Luke Fletcher,et al.  A perception-driven autonomous urban vehicle , 2008 .

[22]  Sebastian Thrun,et al.  Self-supervised Monocular Road Detection in Desert Terrain , 2006, Robotics: Science and Systems.

[23]  Joon Woong Lee,et al.  A Machine Vision System for Lane-Departure Detection , 2002, Comput. Vis. Image Underst..

[24]  Jitendra Malik,et al.  A Comparative Study of Vision-Based Lateral Control Strategies for Autonomous Highway Driving , 1999, Int. J. Robotics Res..

[25]  Christopher M. Kreucher,et al.  LANA: a lane extraction algorithm that uses frequency domain features , 1999, IEEE Trans. Robotics Autom..

[26]  Christopher Rasmussen,et al.  RoadCompass: following rural roads with vision + ladar using vanishing point tracking , 2008, Auton. Robots.

[27]  Jitendra Malik,et al.  A Comparative Study of Vision-Based Lateral Control Strategies for Autonomous Highway Driving , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[28]  C. Kreucher,et al.  A DRIVER WARNING SYSTEM BASED ON THE LOIS LANE DETECTION ALGORITHM , 1998 .

[29]  ZuWhan Kim,et al.  Robust Lane Detection and Tracking in Challenging Scenarios , 2008, IEEE Transactions on Intelligent Transportation Systems.

[30]  Dean A. Pomerleau,et al.  Neural Network Vision for Robot Driving , 1997 .