Hierarchical road understanding for intelligent vehicles based on sensor fusion

Comprehensive situational awareness is paramount to the effectiveness of proprietary navigational and higher-level functions of the intelligent vehicles. In this paper, we address a hierarchical road understanding system for intelligent vehicles with respect to the road topography and the existence of objects based on sensor fusion. The proposed system consists of three modules that run in parallel. Module one classifies the road environment into four categories, i.e. the reachable region, the drivable region, the obstacle region and the unknown region. In module two, an efficient graph-based clustering algorithm is performed in the obstacle region to generate a list of object hypotheses, and their characteristics are used for the coarse identification. In module three, for the object hypotheses in front of the vehicle, particular objects of interest, including vehicles, pedestrians, motorcycles and bicycles, are identified using a multi-class object detector with deformable part-based models, and tracked using particle filters. In the experiments, the data of various typical but challenging road scenarios were acquired by a Velodyne sensor and a monocular camera, and the results have demonstrated the effectiveness of the proposed system.

[1]  Mathias Perrollaz,et al.  Road Scene Analysis by Stereovision: a Robust and Quasi-Dense Approach , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[2]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[3]  Seiichi Mita,et al.  A vision system for autonomous vehicle navigation in challenging traffic scenes using integrated cues , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[4]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[6]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[7]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[8]  David A. McAllester,et al.  Cascade object detection with deformable part models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Martial Hebert,et al.  Onboard contextual classification of 3-D point clouds with learned high-order Markov Random Fields , 2009, 2009 IEEE International Conference on Robotics and Automation.

[10]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[11]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Seiichi Mita,et al.  Graph-based 2D road representation of 3D point clouds for intelligent vehicles , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[13]  Michael Himmelsbach,et al.  Fast segmentation of 3D point clouds for ground vehicles , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[14]  Christoph Stiller,et al.  Segmentation of 3D lidar data in non-flat urban environments using a local convexity criterion , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[15]  Sanjiv Singh,et al.  The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, George Air Force Base, Victorville, California, USA , 2009, The DARPA Urban Challenge.

[16]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..