Off-Road Obstacle Avoidance through End-to-End Learning

We describe a vision-based obstacle avoidance system for off-road mobile robots. The system is trained from end to end to map raw input images to steering angles. It is trained in supervised mode to predict the steering angles provided by a human driver during training runs collected in a wide variety of terrains, weather conditions, lighting conditions, and obstacle types. The robot is a 50cm off-road truck, with two forward-pointing wireless color cameras. A remote computer processes the video and controls the robot via radio. The learning system is a large 6-layer convolutional network whose input is a single left/right pair of unprocessed low-resolution images. The robot exhibits an excellent ability to detect obstacles and navigate around them in real time at speeds of 2 m/s.

[1]  Takeo Kanade,et al.  Vision and Navigation for the Carnegie-Mellon Navlab , 1987 .

[2]  Olivier D. Faugeras,et al.  Maintaining representations of the environment of a mobile robot , 1988, IEEE Trans. Robotics Autom..

[3]  D.J. Kriegman,et al.  Stereo vision and navigation in buildings for mobile robots , 1989, IEEE Trans. Robotics Autom..

[4]  Dean A. Pomerleau,et al.  Knowledge-Based Training of Artificial Neural Networks for Autonomous Robot Driving , 1993 .

[5]  Charles E. Thorpe,et al.  Vision-based neural network road and intersection detection and traversal , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[6]  Martial Hebert,et al.  Mapping and positioning for a prototype lunar rover , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[7]  Erann Gat,et al.  Mars microrover navigation: performance evaluation and enhancement , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  Sebastian Thrun,et al.  Learning Metric-Topological Maps for Indoor Mobile Robot Navigation , 1998, Artif. Intell..

[10]  Alonzo Kelly,et al.  Stereo Vision Enhancements for Low-Cost Outdoor Autonomous Vehicles , 1998 .

[11]  Larry Matthies,et al.  A Compact and Low Power Two-Axis Scanning Laser Rangerfinder for Mobile Robots , 2000 .

[12]  L. Matthies,et al.  A Compact and Low-Power Two-Axis Scanning Laser Rangefinder for Mobile Robots , 2001 .

[13]  Larry Matthies,et al.  Stereo vision and rover navigation software for planetary exploration , 2002, Proceedings, IEEE Aerospace Conference.

[14]  Yann LeCun,et al.  Synergistic Face Detection and Pose Estimation with Energy-Based Models , 2004, J. Mach. Learn. Res..

[15]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..