Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

Recent research on deep learning has been applied to a diversity of fields. In particular, numerous studies have been conducted on self-driving vehicles using end-to-end approaches based on images captured by a single camera. End-to-end controls learn the output vectors of output devices directly from the input vectors of available input devices. In other words, an end-to-end approach learns not by analyzing the meaning of input vectors, but by extracting optimal output vectors based on input vectors. Generally, when end-to-end control is applied to self-driving vehicles, the steering wheel and pedals are controlled autonomously by learning from the images captured by a camera. However, high-resolution images captured from a car cannot be directly used as inputs to Convolutional Neural Networks (CNNs) owing to memory limitations; the image size needs to be efficiently reduced. Therefore, it is necessary to extract features from captured images automatically and to generate input images by merging the parts of the images that contain the extracted features. This paper proposes a learning method for end-to-end control that generates input images for CNNs by extracting road parts from input images, identifying the edges of the extracted road parts, and merging the parts of the images that contain the detected edges. In addition, a CNN model for end-to-end control is introduced. Experiments involving the Open Racing Car Simulator (TORCS), a sustainable computing environment for cars, confirmed the effectiveness of the proposed method for self-driving by comparing the accumulated difference in the angle of the steering wheel in the images generated by it with those of resized images containing the entire captured area and cropped images containing only a part of the captured area. The results showed that the proposed method reduced the accumulated difference by 0.839% and 0.850% compared to those yielded by the resized images and cropped images, respectively.

[1]  Marc Pollefeys,et al.  Motion Estimation for Self-Driving Cars with a Generalized Camera , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Jong Hyuk Park,et al.  Graph-based motor primitive generation framework , 2015, Human-centric Computing and Information Sciences.

[3]  Md. Zainal Abedin,et al.  Traffic sign recognition using hybrid features descriptor and artificial neural network classifier , 2016, 2016 19th International Conference on Computer and Information Technology (ICCIT).

[4]  Raquel Urtasun,et al.  Efficient Deep Learning for Stereo Matching , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Patrick Heinemann,et al.  Context-based detection of pedestrian crossing intention for autonomous driving in urban environments , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Sen Wang,et al.  DeepVO: Towards end-to-end visual odometry with deep Recurrent Convolutional Neural Networks , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Christos Dimitrakakis,et al.  TORCS, The Open Racing Car Simulator , 2005 .

[8]  Rüdiger Dillmann,et al.  Towards a framework for end-to-end control of a simulated vehicle with spiking neural networks , 2016, 2016 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR).

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Changshui Zhang,et al.  Traffic Sign Recognition With Hinge Loss Trained Convolutional Neural Networks , 2014, IEEE Transactions on Intelligent Transportation Systems.

[11]  Jiman Kim,et al.  End-To-End Ego Lane Estimation Based on Sequential Transfer Learning for Self-Driving Cars , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  B. V. K. Vijaya Kumar,et al.  A multi-sensor fusion system for moving object detection and tracking in urban driving environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Vidya N. Murali,et al.  DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Byron Boots,et al.  Agile Off-Road Autonomous Driving Using End-to-End Deep Imitation Learning , 2017, ArXiv.

[16]  Johann Marius Zöllner,et al.  Learning how to drive in a real world simulation with deep Q-Networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[17]  Pascal Vasseur,et al.  Local path planning in a complex environment for self-driving car , 2014, The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent.

[18]  Soyoung Park,et al.  Traffic Information Service Model Considering Personal Driving Trajectories , 2017, J. Inf. Process. Syst..

[19]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[20]  A. H. Mazinan,et al.  On hybrid intelligence-based control approach with its application to flexible robot system , 2017, Human-centric Computing and Information Sciences.

[21]  Sebastian Ramos,et al.  Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling , 2016, 2017 IEEE Intelligent Vehicles Symposium (IV).

[22]  Wenshuo Wang,et al.  Feature analysis and selection for training an end-to-end autonomous vehicle controller using deep learning approach , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[23]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.