Visible and infrared image fusion using an efficient adaptive transition region extraction technique

Abstract In order to track a targeted environment, concealed weapon detection, navigation and military require various imaging modalities, for instance, visible image (VI) and infrared (IR) image. These modalities provide additional details. Complementary information from these images need to be fused into a single image for improved situational awareness. Hence, an ideal fused image should assimilate the essential bright information from the IR image and retain much of the original visual information from the VI. To achieve this, a region based image fusion technique using an efficient adaptive transition region extraction (ATRE) strategy is suggested in this paper. For the first time, the transition region extraction based approach is brought into the context of visible and infrared image fusion. This method is beneficial because it overcomes the problems of noise sensitivity, poor contrast and blurring effects associated with the conventional pixel-based methods. The proposed ATRE technique is used to efficiently extract the bright object regions from the IR image and retain much of the visual background regions from the VI. An adaptive parameter is introduced for accurate segmentation. A region mapping process is followed to get the fused image. Our technique is tested on standard fusion datasets. Image inspection and objective fusion indices are utilized to validate the results. They are compared with conventional and current pixel based and region based fusion techniques. The outcomes reveal that the suggested technique is comparable or better than state-of-the-art fusion techniques.

[1]  Bo Liu,et al.  Multimodal image seamless fusion , 2019, J. Electronic Imaging.

[2]  Ravindra Dhuli,et al.  Two-scale image fusion of visible and infrared images using saliency detection , 2016 .

[3]  Thierry Pun,et al.  A new method for grey-level picture thresholding using the entropy of the histogram , 1980 .

[4]  Xiao-Ping Zhang,et al.  DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion , 2020, IEEE Transactions on Image Processing.

[5]  Zhang Tian,et al.  IMAGE TRANSITION REGION EXTRACTION AND SEGMENTATION BASED ON LOCAL COMPLEXITY , 2005 .

[6]  Tianxu Zhang,et al.  Local entropy-based transition region extraction and thresholding , 2003, Pattern Recognit. Lett..

[7]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[8]  C S Asha,et al.  A novel medical image fusion by combining TV-L1 decomposed textures based on adaptive weighting scheme , 2020 .

[9]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[10]  Ajith Abraham,et al.  A survey on region based image fusion methods , 2019, Inf. Fusion.

[11]  Junjun Jiang,et al.  FusionGAN: A generative adversarial network for infrared and visible image fusion , 2019, Inf. Fusion.

[12]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[13]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[14]  Hui Li,et al.  DenseFuse: A Fusion Approach to Infrared and Visible Images , 2018, IEEE Transactions on Image Processing.

[15]  Gang Liu,et al.  Multi-sensor image fusion based on fourth order partial differential equations , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[16]  K. Ramar,et al.  Spatial frequency discrete wavelet transform image fusion technique for remote sensing applications , 2019, Engineering Science and Technology, an International Journal.

[17]  Vps Naidu,et al.  Image Fusion Technique using Multi-resolution Singular Value Decomposition , 2011 .

[18]  Yu Han,et al.  A new image fusion performance metric based on visual information fidelity , 2013, Inf. Fusion.

[19]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[20]  Hadi Seyedarabi,et al.  A non-reference image fusion metric based on mutual information of image features , 2011, Comput. Electr. Eng..

[21]  Wei Yu,et al.  Infrared and visible image fusion via detail preserving adversarial learning , 2020, Inf. Fusion.

[22]  Sun Li,et al.  Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters , 2016, Inf. Fusion.

[23]  Jiangshe Zhang,et al.  DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion , 2020, IJCAI.

[24]  David Zhang,et al.  Modified local entropy-based transition region extraction and thresholding , 2011, Appl. Soft Comput..

[25]  Durga Prasad Bavirisetti,et al.  Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve Transform , 2016, IEEE Sensors Journal.

[26]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[27]  Xin Jin,et al.  Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain , 2018 .

[28]  Hua Zhao,et al.  Visible and infrared image fusion based on Curvelet transform , 2014, The 2014 2nd International Conference on Systems and Informatics (ICSAI 2014).

[29]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[30]  Shaowen Yao,et al.  A survey of infrared and visual image fusion methods , 2017 .

[31]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.

[32]  Zhongliang Jing,et al.  Review of pixel-level image fusion , 2010 .

[33]  Yu Zhang,et al.  Infrared and visual image fusion through infrared feature extraction and visual information preservation , 2017 .

[34]  Wencheng Wang,et al.  A Multi-focus Image Fusion Method Based on Laplacian Pyramid , 2011, J. Comput..

[35]  Nikolaos Mitianoudis,et al.  Region-based ICA image fusion using textural information , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[36]  Nikolaos Mitianoudis,et al.  Pixel-based and region-based image fusion schemes using ICA bases , 2007, Inf. Fusion.

[37]  Shutao Li,et al.  Hybrid Multiresolution Method for Multisensor Multimodal Image Fusion , 2010, IEEE Sensors Journal.

[38]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[39]  Josef Kittler,et al.  Infrared and Visible Image Fusion using a Deep Learning Framework , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).