Layering Laser Rangefinder Points onto Images for Obstacle Avoidance by a Neural Network

Obstacle avoidance is essential to autonomous robot navigation but maneuvering around an obstacle causes the system to deviate from its normal path. Oftentimes, these deviations cause the robot to enter new regions which lack the path’s usual or meaningful features. This is problematic for vision-based steering controllers, including convolutional neural networks (CNNs), which depend on patterns to be present in camera images. The absence of a path fails to provide consistent and noticeable patterns for the neural network, and this usually leads to erroneous steering commands. In this paper, we mitigate this problem by superimposing points from a two-dimensional (2D) scanning laser rangefinder (LRF) onto camera images using the Open Source Computer Vision (OpenCV) library. The visually encoded LRF data provides the CNN with a new pattern to recognize, aiding in the avoidance of obstacles and the rediscovery of its path. In contrast, existing approaches to robot navigation do not use a single CNN to perform line-following and obstacle avoidance. Using our approach, we were able to train a CNN to follow a lined path and avoid obstacles with a reliability rate of nearly 60% on a complex course and over 80% on a simpler course.

[1]  Hong Yu,et al.  Obstacle Detection with Deep Convolutional Neural Network , 2013, 2013 Sixth International Symposium on Computational Intelligence and Design.

[2]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[3]  Christopher J. Lowrance,et al.  Application of Convolutional Neural Network Image Classification for a Path-Following Robot , 2018, 2018 IEEE MIT Undergraduate Research Technology Conference (URTC).

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Sen Wang,et al.  Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning , 2017, RSS 2017.

[6]  Christopher J. Lowrance,et al.  Smoother Robot Control from Convolutional Neural Networks Using Fuzzy Logic , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[7]  Keith Sullivan,et al.  Deep Obstacle Avoidance ∗ , 2016 .

[8]  Chunyang Wang,et al.  CNN-Based Vision Model for Obstacle Avoidance of Mobile Robot , 2017 .

[9]  Illah R. Nourbakhsh,et al.  Appearance-Based Obstacle Detection with Monocular Color Vision , 2000, AAAI/IAAI.

[10]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[11]  Kahlouche Souhila,et al.  Optical Flow Based Robot Obstacle Avoidance , 2007 .

[12]  Tian Xia,et al.  Vehicle Detection from 3D Lidar Using Fully Convolutional Network , 2016, Robotics: Science and Systems.

[13]  Daniel Göhring,et al.  Application of open Source Deep Neural Networks for Object Detection in Industrial Environments , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[14]  Toshiyuki Yasuda,et al.  Development of robot navigation method based on single camera vision using deep learning , 2017, 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).