Off-road path and obstacle detection using monocular camera

Advanced Driver Assistance Systems (ADAS) are becoming more common. Most ADAS systems are designed for paved roads, and rely on the road markings to detect and alert hazards. However, road aware systems, suitable for off-road and dirt roads are less common. This work presents a single camera-based non-paved road identification and obstacle detection and alert system, where the road has neither lane nor shoulders markings. It can be used either as part of an ADAS system, or for autonomous vehicle, suitable for dirt roads. The system utilizes a single standard monocular consumer grade smartphone's camera sensor as its input. Furthermore, the work presented here is robust in the sense it is neither camera specific, nor vehicle specific, and therefore may be adapted to any type of standard camera or vehicle. The presented algorithm combines two-dimensional (2D) and three-dimensional (3D) video analysis techniques. Structure from Motion and Visual Odometry techniques are used to convert the 2D image sequence into a 3D point cloud model. This model is used to estimate and detect nearby obstacles and traversable terrain along a driven off-road path. Usage of 2D image processing techniques such as edge detection and color segmentation, allow reliable estimation of distant parts of traversable terrain road. Their combination provides robust point cloud creation combined with low computational cost, thus enabling near real time operation.

[1]  Antonio M. López,et al.  Road Detection Based on Illuminant Invariance , 2011, IEEE Transactions on Intelligent Transportation Systems.

[2]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[3]  Eyal Katz,et al.  Automatic parking identification and vehicle guidance with road awareness , 2016, 2016 IEEE International Conference on the Science of Electrical Engineering (ICSEE).

[4]  E. Katz,et al.  A single camera based floating virtual keyboard with improved touch detection , 2012, 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel.

[5]  Andrew E. Johnson,et al.  Computer Vision on Mars , 2007, International Journal of Computer Vision.

[6]  Hermann Winner,et al.  Three Decades of Driver Assistance Systems: Review and Future Perspectives , 2014, IEEE Intelligent Transportation Systems Magazine.

[7]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[8]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[9]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Urs A. Muller,et al.  Learning long-range vision for autonomous off-road driving , 2009 .

[12]  Sei-Wang Chen,et al.  Shadow detection and removal for traffic images , 2004, IEEE International Conference on Networking, Sensing and Control, 2004.

[13]  Philip H. S. Torr,et al.  Combining Appearance and Structure from Motion Features for Road Scene Understanding , 2009, BMVC.

[14]  Theo Gevers,et al.  3D Scene priors for road detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[16]  D. Viswanathan,et al.  Features from Accelerated Segment Test ( FAST ) , 2011 .

[17]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[18]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[19]  Sebastian Thrun,et al.  Self-supervised Monocular Road Detection in Desert Terrain , 2006, Robotics: Science and Systems.

[20]  Martin Lauer,et al.  A generative model for 3D urban scene understanding from movable platforms , 2011, CVPR 2011.

[21]  C. V. Jawahar,et al.  Scene Text Recognition using Higher Order Language Priors , 2009, BMVC.

[22]  Yann LeCun,et al.  Online Learning for Offroad Robots: Spatial Label Propagation to Learn Long-Range Traversability , 2007, Robotics: Science and Systems.

[23]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[24]  Jaesang Lee,et al.  Road Following in an Unstructured Desert Environment Based on the EM(Expectation-Maximization) Algorithm , 2006, 2006 SICE-ICASE International Joint Conference.