Investigation of image fusion algorithms and performance evaluation for night vision

Image fusion is a technique used to combine multiple source images of the same scene to obtain a more complete and accurate view at the scene. Numerous image fusion algorithms have been developed in the past years. The fusion approaches range in complexity from those which are extremely simple to those which are extremely complex. Our investigation focuses on the fusion algorithms and quality measures for the night vision applications. To judge the performance of image fusion algorithms, both subjective and objective evaluation approaches are investigated. The preliminary human evaluations indicate that the observers generally prefer the Shift Invariant Discrete Wavelet Trans-form (SiDWT) and Laplacian fusion methods over the other methods. Furthermore, to evaluate image fusion algorithms objectively, we investigated a few image quality measures which do not require the availability of an ideal image. The tests of objective quality measures show that the edge based metric (QE) better matches human evaluations than do the other methods for the images we considered. While recently a few image fusion quality measures have been proposed, analytical studies of them have been lacking. We focus on one popular mutual information-based quality measure. Based on an image formation model, we obtain a closed-form expression for the quality measure and mathematically analyze its properties under different types of image distortion. Tests with real images are also presented which agree with the conclusions of the analytical results. Automated image quality assessment is highly desirable to evaluate the performance of various image fusion algorithms for night vision applications. We propose a perceptual quality evaluation method for image fusion which is based on human visual system (HVS) models. The most popular existing algorithms are also evaluated. For some specific parameter settings, we find our algorithm provides better predictions, which are more closely matched to human perceptual evaluations, than the existing algorithms. The evaluations focus on the night vision application, but the algorithm we propose is applicable to other applications also. The quality of the input images is a major factor affecting the performance of Automatic Target Recognition (ATR) systems. It is desirable to know at what quality level the input images would most likely cause ATR system failure. If one can correlate the algorithm performance with different image quality measures, the recognition confidence can be predicted before applying ATR by predetermining the input image quality. Image quality based prediction also has value for image fusion, since the prediction can help to determine whether alternative images should be captured for fusion. We address the utility of image quality measures and their correlations with performance failures of a principle component analysis (PCA) based ATR algorithm. Various image fusion approaches are examined to illustrate their abilities to improve ATR performance. Results show that the Shift Invariant Discrete Wavelet Transform (SiDWT) and Laplacian pyramid fusion schemes outperform other methods for improving the detection rate with the considered Synthetic Aperture Radar (SAR) images. Regression analysis is conducted to show that linear combinations of the selected image quality measures could explain about 60% of the variability in the non-detections of the ATR algorithm.