3D road curb extraction from image sequence for automobile parking assist system

We extract 3D curb from video sequence, using a single camera equipped with fish-eye lens and located at the front/rear of the vehicle. The challenge in extracting curbs from images lies in their small size and their lack of texture. We show that by appropriately exploiting appearance features, 3D geometry, and temporal information, one can reliably detect and localize the curbs in the 3D scene. The main underlying assumption of our model is that the road surface is flat and that the curb is approximately orthogonal to the road plane. We collected nine videos with ground truth, under day-time sunny weather condition, up to 2m range. Our experimental results compare favorably wrt the current the state-of-the-art on our database -90% precision rate in average and over 85% accuracy in curb height estimation.

[1]  Roberto Manduchi,et al.  Visual curb localization for autonomous navigation , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[2]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[3]  Sanja Fidler,et al.  Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Raúl Rojas,et al.  Camera based detection and classification of soft shoulders, curbs and guardrails , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[5]  Roberto Manduchi,et al.  Robust curb and ramp detection for safe parking using the Canesta TOF camera , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Ulrich Hofmann,et al.  Multi-cue, Model-Based Detection and Mapping of Road Curb Features Using Stereo Vision , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[7]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  R. Aufrere,et al.  Multiple sensor fusion for detecting location of curbs, walls, and barriers , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[9]  Slobodan Ilic,et al.  Scene understanding from a moving camera for object detection and free space estimation , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[10]  Roland Siegwart,et al.  A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[11]  Stephen Gould,et al.  Region-based Segmentation and Object Detection , 2009, NIPS.

[12]  Tadahiro Kuroda,et al.  Adaptive window search using semantic texton forests for real-time object detection , 2013, 2013 IEEE International Conference on Image Processing.

[13]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[14]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[15]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[16]  Sanja Fidler,et al.  Holistic 3D scene understanding from a single geo-tagged image , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jonathan Krause,et al.  Leveraging the Wisdom of the Crowd for Fine-Grained Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Xiaojin Gong,et al.  Fusion Based Holistic Road Scene Understanding , 2014, ArXiv.

[19]  Wolfgang Förstner,et al.  Curb reconstruction using Conditional Random Fields , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[20]  Sergiu Nedevschi,et al.  Curb detection for driving assistance systems: A cubic spline-based approach , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[21]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.