Obstacle Detection and Safely Navigate the Autonomous Vehicle from Unexpected Obstacles on the Driving Lane

Nowadays, autonomous vehicle is an active research area, especially after the emergence of machine vision tasks with deep learning. In such a visual navigation system for autonomous vehicle, the controller captures images and predicts information so that the autonomous vehicle can safely navigate. In this paper, we first introduced small and medium-sized obstacles that were intentionally or unintentionally left on the road, which can pose hazards for both autonomous and human driving situations. Then, we discuss Markov random field (MRF) model by fusing three potentials (gradient potential, curvature prior potential, and depth variance potential) to segment the obstacles and non-obstacles into the hazardous environment. Since the segment of obstacles is done by MRF model, we can predict the information to safely navigate the autonomous vehicle form hazardous environment on the roadway by DNN model. We found that our proposed method can segment the obstacles accuracy from the blended background road and improve the navigation skills of the autonomous vehicle.

[1]  Véronique Berge-Cherfaoui,et al.  Vehicle trajectory prediction based on motion model and maneuver recognition , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  R. Aufrere,et al.  Multiple sensor fusion for detecting location of curbs, walls, and barriers , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[3]  Ahmad Reza Naghsh-Nilchi,et al.  High-order Markov random field for single depth image super-resolution , 2017, IET Comput. Vis..

[4]  T. Williamson.  A high-performance stereo vision system for obstacle detection , 1998 .

[5]  Kok Kiong Tan,et al.  Vision-based lane line detection for autonomous vehicle navigation and guidance , 2015, 2015 10th Asian Control Conference (ASCC).

[6]  H. Toyoda,et al.  A 3.2 kHz, 14-Bit Optical Absolute Rotary Encoder With a CMOS Profile Sensor , 2008, IEEE Sensors Journal.

[7]  Sebastian Ramos,et al.  Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling , 2016, 2017 IEEE Intelligent Vehicles Symposium (IV).

[8]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[9]  Sergiu Nedevschi,et al.  Polynomial curb detection based on dense stereovision for driving assistance , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[10]  Mehmet Türkan,et al.  Autonomous Cars: Vision based Steering Wheel Angle Estimation , 2019, ArXiv.

[11]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[12]  Stan Z. Li,et al.  Markov Random Field Modeling in Image Analysis , 2001, Computer Science Workbench.

[13]  Brian C Tefft,et al.  The Prevalence of Motor Vehicle Crashes Involving Road Debris, United States, 2011-2014 , 2016 .

[14]  Markus H. Gross,et al.  Efficient simplification of point-sampled surfaces , 2002, IEEE Visualization, 2002. VIS 2002..

[15]  F. Chausse,et al.  A fast and robust vision based road following algorithm , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[16]  Qiao Sun,et al.  TOF Lidar Development in Autonomous Vehicle , 2018, 2018 IEEE 3rd Optoelectronics Global Conference (OGC).

[17]  Jun Tan,et al.  Robust Curb Detection with Fusion of 3D-Lidar and Camera Data , 2014, Sensors.

[18]  Rudolf Mester,et al.  High-performance long range obstacle detection using stereo vision , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Sergiu Nedevschi,et al.  Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection , 2010, IEEE Transactions on Vehicular Technology.

[21]  Bogdan Savchynskyy,et al.  Discrete Graphical Models - An Optimization Perspective , 2019, Found. Trends Comput. Graph. Vis..

[22]  Zhitao Xiao,et al.  Combining CNN and MRF for road detection , 2017, Comput. Electr. Eng..

[23]  Massimo Bertozzi,et al.  Real-time obstacle detection using stereo vision for autonomous ground vehicles: A survey , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[24]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[25]  Daniel Cremers,et al.  B-Spline Modeling of Road Surfaces With an Application to Free-Space Estimation , 2009, IEEE Transactions on Intelligent Transportation Systems.

[26]  Jitendra Malik,et al.  An integrated stereo-based approach to automatic vehicle guidance , 1995, Proceedings of IEEE International Conference on Computer Vision.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Alberto Broggi,et al.  A full-3D voxel-based dynamic obstacle detection for urban scenario using stereo vision , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[29]  Fernando Santos Osório,et al.  Robust curb detection and vehicle localization in urban environments , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[30]  Hanna Jeppsson,et al.  Real life safety benefits of increasing brake deceleration in car-to-pedestrian accidents: Simulation of Vacuum Emergency Braking. , 2018, Accident; analysis and prevention.

[31]  Baoxin Li,et al.  Robust Ground Plane Detection with Normalized Homography in Monocular Sequences from a Robot Platform , 2006, 2006 International Conference on Image Processing.

[32]  Alberto Elfes,et al.  Using occupancy grids for mobile robot perception and navigation , 1989, Computer.

[33]  C. Rabe,et al.  Fast detection of moving objects in complex scenarios , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[34]  Alberto Broggi,et al.  Stereo obstacle detection in challenging environments: The VIAC experience , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Roberto Manduchi,et al.  Obstacle Detection and Terrain Classification for Autonomous Off-Road Navigation , 2005, Auton. Robots.

[36]  Yeong-Kang Lai,et al.  Intelligent Vehicle Collision-Avoidance System with Deep Learning , 2018, 2018 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[37]  Dewant Katare,et al.  Embedded System Enabled Vehicle Collision Detection: An ANN Classifier , 2019, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC).

[38]  Dong Eui Chang,et al.  Cooperative Adaptive Cruise Control Using Turn Signal for Smooth and Safe Cut-In , 2018, 2018 18th International Conference on Control, Automation and Systems (ICCAS).

[39]  H. Toyoda,et al.  A 3.2 kHz, 13-bit Optical Absolute Rotary Encoder with a CMOS Profile Sensor , 2007, 2007 IEEE Sensors.

[40]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[41]  Nikos Komodakis,et al.  Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey , 2013, Comput. Vis. Image Underst..

[42]  D. Pham,et al.  Statistical approach to normalization of feature vectors and clustering of mixed datasets , 2012, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[43]  Romuald Aufrère,et al.  A model-driven approach for real-time road recognition , 2001, Machine Vision and Applications.

[44]  Luc Van Gool,et al.  Object Detection and Tracking for Autonomous Navigation in Dynamic Environments , 2010, Int. J. Robotics Res..

[45]  Mashrur Chowdhury,et al.  Risk Analysis of Autonomous Vehicles in Mixed Traffic Streams , 2017 .

[46]  Jiebo Luo,et al.  End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[47]  Sebastian Ramos,et al.  Lost and Found: detecting small road hazards for self-driving vehicles , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[48]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Edmondo Trentin,et al.  Hybrid Random Fields - A Scalable Approach to Structure and Parameter Learning in Probabilistic Graphical Models , 2011, Intelligent Systems Reference Library.

[50]  Xin Chen,et al.  High-Definition Digital Elevation Model System Vision Paper , 2017, SSDBM.

[51]  M. Fukushima,et al.  Levenberg–Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints , 2004 .

[52]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Darius Burschka,et al.  Efficient occupancy grid computation on the GPU with lidar and radar for road boundary detection , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[54]  David Fernández Llorca,et al.  Curvature-based curb detection method in urban environments using stereo and laser , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[55]  Danya Yao,et al.  Integrated Variable Speed Limits and Lane-Changing Control for Freeway Lane-Drop Bottlenecks , 2020, IEEE Access.

[56]  John K. Tsotsos,et al.  50 Years of object recognition: Directions forward , 2013, Comput. Vis. Image Underst..

[57]  Sebastian Thrun,et al.  Upsampling range data in dynamic environments , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[58]  Xian Xu,et al.  Autonomous vehicles, risk perceptions and insurance demand: An individual survey in China , 2019, Transportation Research Part A: Policy and Practice.

[59]  Mario A. López,et al.  STR: a simple and efficient algorithm for R-tree packing , 1997, Proceedings 13th International Conference on Data Engineering.

[60]  S. Poczter,et al.  The Google Car: Driving Toward A Better Future? , 2013 .

[61]  Uwe Franke,et al.  Towards a Global Optimal Multi-Layer Stixel Representation of Dense 3D Data , 2011, BMVC.

[62]  Jannik Fritsch,et al.  A self-adaptive approach for curbstone/roadside detection based on human-like signal processing and multi-sensor fusion , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[63]  Ladislau Bölöni,et al.  Reducing Side-Sweep Accidents with Vehicle-to-Vehicle Communication , 2016, J. Sens. Actuator Networks.

[64]  John Hancock.  High-Speed Obstacle Detection for Automated Highway Applications , 1997 .

[65]  Ram Gopal Raj,et al.  Street Sign Recognition Using Histogram of Oriented Gradients and Artificial Neural Networks , 2019, J. Imaging.

[66]  Antonin Guttman,et al.  R-trees: a dynamic index structure for spatial searching , 1984, SIGMOD '84.

[67]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[68]  Aamer Iqbal Bhatti,et al.  Trajectory Planning and Control for Lane-Change of Autonomous Vehicle , 2019, 2019 5th International Conference on Control, Automation and Robotics (ICCAR).

[69]  Yassine Ruichek,et al.  Building variable resolution occupancy grid map from stereoscopic system — A quadtree based approach , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[70]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[71]  Y. Ninomiya,et al.  Model-based lane recognition , 1996, Proceedings of Conference on Intelligent Vehicles.

[72]  Yann LeCun,et al.  Learning long‐range vision for autonomous off‐road driving , 2009, J. Field Robotics.

[73]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[74]  J. Goldbeck,et al.  Lane detection and tracking by video sensors , 1999, Proceedings 199 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems (Cat. No.99TH8383).

[75]  Julius Ziegler,et al.  Sparse scene flow segmentation for moving object detection in urban environments , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[76]  K. Madhava Krishna,et al.  Markov Random Field based small obstacle discovery over images , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).