Obstacle detection and classification using deep learning for tracking in high-speed autonomous driving

On-road obstacle detection and classification is one of the key tasks in the perception system of self-driving vehicles. Since vehicle tracking involves localizationand association of vehicles between frames, detection and classification of vehicles is necessary. Vision-based approaches are popular for this task due to cost-effectiveness and usefulness of appearance information associated with the vision data. In this paper, a deep learning system using region-based convolutional neural network trained with PASCAL VOC image dataset is developed for the detection and classification of on-road obstacles such as vehicles, pedestrians and animals. The implementation of the system on a Titan X GPU achieves a processing frame rate of at least 10 fps for a VGA resolution image frame. This sufficiently high frame rate using a powerful GPU demonstrate the suitability of the system for highway driving of autonomous cars. The detection and classification results on images from KITTI and iRoads, and also Indian roads show the performance of the system invariant to object's shape and view, and different lighting and climatic conditions.