Simultaneous detection of lane and pavement boundaries using model-based multisensor fusion

Treats a problem arising in the design of intelligent vehicles: automated detection of lane and pavement boundaries using forward-looking optical and radar imaging sensors mounted on an automobile. In previous work, lane and pavement boundaries have always been located separately. This separate detection strategy is problematic in situations when either the optical or the radar image is too noisy. We propose a Bayesian multisensor image fusion method to solve our boundary detection problem. This method makes use of a deformable template model to globally describe the boundaries of interest. The optical and radar imaging processes are described with random field likelihoods. The multisensor fusion boundary detection problem is reformulated as a joint MAP estimation problem. However, the joint MAP estimate is intractable, as it involves the computation of a notoriously difficult normalization constant, also known as the partition function. Therefore, we settle for the so-called empirical MAP estimate, as an approximation to the true MAP estimate. Several experimental results are provided to demonstrate the efficacy of the empirical MAP estimation method in simultaneously detecting lane and pavement boundaries. Fusion of multi-modal images is not only of interest to the intelligent vehicles community, but to others as well, such as biomedicine, remote sensing, target recognition. The method presented in the paper is also applicable to image fusion problems in these other areas.

[1]  Surender K Kenue,et al.  Lanelok: Detection Of Lane Boundaries And Vehicle Tracking Using Image-Processing Techniques - Part I: Hough-Transform, Region-Tracing And Correlation Algorithms , 1989, Other Conferences.

[2]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[3]  Robinson Piramuthu,et al.  Minimax Emission Computed Tomography using High-Resolution Anatomical Side Information and B-Spline Models , 1999, IEEE Trans. Inf. Theory.

[4]  Khursheed Alam,et al.  An empirical bayes estimate of multinomial probabilities , 1986 .

[5]  Sridhar Lakshmanan,et al.  Building world models for mobile platforms using heterogeneous sensors fusion and temporal analysis , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[6]  Karl C. Kluge.  Performance evaluation of vision-based lane sensing: some preliminary tools, metrics, and results , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[7]  Nan M. Laird,et al.  Empirical bayes estimates using the nonparametric maximum likelihood estimate for the priort , 1982 .

[8]  Todd Jochem,et al.  Rapidly Adapting Machine Vision for Automated Vehicle Steering , 1996, IEEE Expert.

[9]  Sridhar Lakshmanan,et al.  An algorithm for detecting roads and obstacles in radar images , 2001, IEEE Trans. Veh. Technol..

[10]  Sridhar Lakshmanan,et al.  A deformable-template approach to lane detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[11]  Robert C. Harney.  Sensor fusion for target recognition: a review of fundamentals and a potential approach to multisensor requirements allocation , 1994, Optics & Photonics.

[12]  Dirk Borghys,et al.  Long-range target detection in a cluttered environment using multisensor image sequences , 1997, Defense, Security, and Sensing.

[13]  S. B. Serpico,et al.  Fusion of multisensor and multitemporal data in remote-sensing image analysis , 1998, IGARSS '98. Sensing and Managing the Environment. 1998 IEEE International Geoscience and Remote Sensing. Symposium Proceedings. (Cat. No.98CH36174).

[14]  Alfonso Farina,et al.  Coherent radar detection in log-normal clutter , 1986 .

[15]  Dean A. Pomerleau,et al.  The 1997 automated highway free agent demonstration , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[16]  M. Skolnik,et al.  Introduction to Radar Systems , 2021, Advances in Adaptive Radar Detection and Range Estimation.

[17]  S. Iyengar,et al.  Multi-Sensor Fusion: Fundamentals and Applications With Software , 1997 .

[18]  Karl Kluge.  YARF: An Open-Ended Framework for Robot Road Following , 1993 .

[19]  D. Langer,et al.  Fusing radar and vision for detecting, classifying and avoiding roadway obstacles , 1996, Proceedings of Conference on Intelligent Vehicles.

[20]  Alfred O. Hero,et al.  Detection of curved road edges in radar images via deformable templates , 1997, Proceedings of International Conference on Image Processing.

[21]  Jong-Sen Lee,et al.  Speckle analysis and smoothing of synthetic aperture radar images , 1981 .

[22]  Anil K. Jain,et al.  Detecting straight edges in millimeter-wave images , 1995, Proceedings., International Conference on Image Processing.

[23]  O. Basset,et al.  Fusion of ultrasonic and radiographic images of the breast , 1996, 1996 IEEE Ultrasonics Symposium. Proceedings.

[24]  T. Kozek,et al.  Road lane marker extraction by motion-detector CNNs , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[25]  Sridhar Lakshmanan,et al.  CLARK: a heterogeneous sensor fusion method for finding lanes and obstacles , 2000, Image Vis. Comput..

[26]  Sridhar Lakshmanan,et al.  Lane detection for automotive sensors , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[27]  Sridhar Lakshmanan,et al.  A Deformable Template Approach to Detecting Straight Edges in Radar Images , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ernst D. Dickmanns,et al.  Recursive 3-D Road and Relative Ego-State Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Barbara T. Sweet,et al.  Image processing and fusion for landing guidance , 1996, Defense, Security, and Sensing.

[30]  Alain Hillion,et al.  An information fusion method for multispectral image classification postprocessing , 1998, IEEE Trans. Geosci. Remote. Sens..

[31]  Karl C. Kluge,et al.  Extracting road curvature and orientation from image edge points without perceptual grouping into features , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.