Road obstacles positional and dynamic features extraction combining object detection, stereo disparity maps and optical flow data

One of the most relevant tasks in an intelligent vehicle navigation system is the detection of obstacles. It is important that a visual perception system for navigation purposes identifies obstacles, and it is also important that this system can extract essential information that may influence the vehicle’s behavior, whether it will be generating an alert for a human driver or guide an autonomous vehicle in order to be able to make its driving decisions. In this paper we present an approach for the identification of obstacles and extraction of class, position, depth and motion information from these objects that employs data gained exclusively from passive vision. We use a convolutional neural network for the obstacles detection, optical flow for the analysis of movement of the detected obstacles, both in relation to the direction and in relation to the intensity of the movement, and also stereo vision for the analysis of distance of obstacles in relation to the vehicle. We performed our experiments on two different datasets, and the results obtained showed a good efficacy from the use of depth and motion patterns to assess the obstacles’ potential threat status.

[1]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Sandeep Singh Sengar,et al.  Motion segmentation-based surveillance video compression using adaptive particle swarm optimization , 2019, Neural Computing and Applications.

[3]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yingping Huang,et al.  Obstacle recognition for ADAS using stereovision and snake models , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[5]  Mohan M. Trivedi,et al.  How Would Surround Vehicles Move? A Unified Framework for Maneuver Classification and Motion Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[6]  Sandeep Singh Sengar,et al.  Motion detection using block based bi-directional optical flow method , 2017, J. Vis. Commun. Image Represent..

[7]  David J. Fleet,et al.  Optical Flow Estimation , 2006, Handbook of Mathematical Models in Computer Vision.

[8]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Fernando Santos Osório,et al.  CaRINA Intelligent Robotic Car: Architectural design and applications , 2014, J. Syst. Archit..

[10]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[11]  N. B. Puhan,et al.  Adaptive saliency-weighted obstacle detection for the visually challenged , 2015, 2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN).

[12]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[13]  Bastian Leibe,et al.  Real Time Vision Based Multi-person Tracking for Mobile Robotics and Intelligent Vehicles , 2011, ICIRA.

[14]  Sandeep Singh Sengar,et al.  Moving object detection using statistical background subtraction in wavelet compressed domain , 2019, Multimedia Tools and Applications.

[15]  Jing Li,et al.  On-Road Multiple Obstacles Detection in Dynamical Background , 2014, 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics.

[16]  Ta-Te Lin,et al.  Road environment recognition method in complex traffic situations based on stereo vision , 2012, 2012 12th International Conference on ITS Telecommunications.

[17]  Chang Wen Chen,et al.  A novel video analysis approach for overtaking vehicle detection , 2013, 2013 International Conference on Connected Vehicles and Expo (ICCVE).

[18]  Aldo von Wangenheim,et al.  Passive vision road obstacle detection: a literature mapping , 2020, International Journal of Computers and Applications.

[19]  Fernando Santos Osório,et al.  CaRINA dataset: An emerging-country urban scenario benchmark for road detection systems , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[20]  Yingping Huang,et al.  Multi-Class Obstacle Detection and Classification Using Stereovision and Improved Active Contour Models , 2016 .

[21]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[22]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[23]  K. Madhava Krishna,et al.  Small obstacle detection using stereo vision for autonomous ground vehicle , 2017, AIR '17.

[24]  Sergiu Nedevschi,et al.  Superpixel-based obstacle segmentation from dense stereo urban traffic scenarios using intensity, depth and optical flow information , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[25]  Sandeep Singh Sengar,et al.  Moving object detection based on frame difference and W4 , 2017, Signal Image Video Process..

[26]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[29]  Honghai Liu,et al.  Intelligent Robotics and Applications , 2014, Lecture Notes in Computer Science.

[30]  Rajesh Kumar,et al.  Obstacle detection and classification using deep learning for tracking in high-speed autonomous driving , 2017, 2017 IEEE Region 10 Symposium (TENSYMP).

[31]  Lars Schmidt-Thieme,et al.  Data-Driven Vehicle Trajectory Forecasting , 2019, ArXiv.

[32]  Bertrand Zavidovique,et al.  Obstacle detection ”for free” in the c-velocity space , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[33]  Torsten Sattler,et al.  3D visual perception for self-driving cars using a multi-camera system: Calibration, mapping, localization, and obstacle detection , 2017, Image Vis. Comput..

[34]  Wesley J. Marshall.  Update on the ANSI standard for safe use of lasers outdoors , 2005 .

[35]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[36]  Yu Yao,et al.  Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systems , 2018, 2019 International Conference on Robotics and Automation (ICRA).