A knowledge-based neural network for fusing edge maps of multi-sensor images

With the goal of fusion prescribed as building an edge map that contains as many edges as possible from the given multi-spectral/sensor images, a new fusion scheme, called the knowledge-based neural network fusion (KBNNF), is proposed to fuse edge maps of these images in order to generate a combined edge map that has more complete and reliable edge information than what one can obtain from any single image. The KBNNF is used to fuse edge maps of images having mutually complementary edge information in the following sense: (i) the edges in the images are compatible, i.e., can be interpreted together; and (ii) the edges in the different images reveal different parts of the scene. More complete edge contours of the same object are obtained by linking the edge sections obtained from different images together. The resulting edge map can be used for subsequent study (like object recognition). The proposed scheme bases its confidence and reliability on the analysis of variance (ANOVA)-based edge detector that can address two important issues of edge based image fusion well: (i) the difference in edge position among the images because of the different characteristics of the images and the error in the image registration process; and (ii) the variance existing among the edge test values calculated from different images. The KBNNF has been applied to fuse: (i) radar (SAR)–optical (SPOT), (ii) optical–optical, (iii) infrared–infrared, and (iv) optical–infrared (satellite) image combinations. Comparisons are made with the relevant existing techniques in the literature. The paper concludes with some examples to illustrate the efficacy of the proposed scheme.

[1]  Alexander Toet,et al.  Merging thermal and visual images by a contrast pyramid , 1989 .

[2]  P. Atkinson,et al.  Introduction Neural networks in remote sensing , 1997 .

[3]  Lawrence A. Klein,et al.  Sensor and Data Fusion Concepts and Applications , 1993 .

[4]  Kuo-Chu Chang,et al.  Polarimetric fusion for synthetic aperture radar target classification , 1997, Pattern Recognit..

[5]  Xavier Otazu,et al.  Image fusion with additive multiresolution wavelet decomposition. Applications to SPOT+Landsat images , 1999 .

[6]  Ludwik Kurz,et al.  Analysis of Variance in Statistical Image Processing , 1997 .

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  D. L. Hall,et al.  Mathematical Techniques in Multisensor Data Fusion , 1992 .

[9]  Xiaoou Tang,et al.  Multiple competitive learning network fusion for object classification , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Y. V. Venkatesh,et al.  Multisensor image fusion using influence factor modification and the ANOVA methods , 2000, IEEE Trans. Geosci. Remote. Sens..

[11]  D. Yocky.  Image merging and data fusion by means of the discrete two-dimensional wavelet transform , 1995 .

[12]  Alain Hillion,et al.  An information fusion method for multispectral image classification postprocessing , 1998, IEEE Trans. Geosci. Remote. Sens..

[13]  Anil K. Jain,et al.  A Markov random field model for classification of multisource satellite imagery , 1996, IEEE Trans. Geosci. Remote. Sens..

[14]  James Llinas,et al.  An introduction to multi-sensor data fusion , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[15]  Laurent Mascarilla,et al.  Combination of remote sensing and geocoded data for satellite image interpretation based on neural networks , 1993, Proceedings of IGARSS '93 - IEEE International Geoscience and Remote Sensing Symposium.

[16]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[17]  Li-Min Fu.  Knowledge-based connectionism for revising domain theories , 1993, IEEE Trans. Syst. Man Cybern..

[18]  Belur V. Dasarathy.  Adaptive fusion processor paradigms for fusion of information acquired at different levels of detail , 1996 .

[19]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[20]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[21]  Alexander Toet,et al.  New false color mapping for image fusion , 1996 .

[22]  Robert A. Schowengerdt,et al.  A review and analysis of backpropagation neural networks for classification of remotely-sensed multi-spectral imagery , 1995 .

[23]  V. K. Shettigara,et al.  A generalized component substitution technique for spatial enhancement of multispectral images using , 1992 .

[24]  A. Farina,et al.  The fusion of different resolution SAR images , 1997 .