Adaptive selection of visual and infra-red image fusion rules

The fusion of images captured from Electrical-Optical (EO) and Infra-Red (IR) cameras has been extensively studied for military applications in recent years. In this paper, we propose a novel wavelet-based framework for online fusion of EO and IR image sequences. The proposed framework provides multiple fusion rules for image fusion as well as a novel edge-based evaluation method to select the optimal fusion rule with respect to different practical scenarios. In the fusion step, EO and IR images are decomposed into different levels by 2D discrete wavelet transform. The wavelet coefficients at each level are combined by a set of fusion rules, such as min-max selection, mean-value, weighted summations, etc. Various fused images are obtained by inverse wavelet transform of combined coefficients. In the evaluation step, Sobel operator is applied on both the fused images and original images. Compared with original images, the remaining edge information in the fused each image is calculated as the fusion quality assessment. Finally, the fused image with the highest assessment value will be selected as the fusion result. In addition, the proposed method can adaptively select the best fusion rule for EO and IR images under different scenarios.