Contrast-enhanced fusion of infrared and visible images

Abstract. The fusion of infrared and visible images may result in low contrast, which is unsuitable for observation by human eyes. Thus, we propose a contrast-enhanced fusion algorithm with nonsubsampled shearlet transform (NSST) frames, in which the NSST is first employed to decompose each of the source images into one low frequency sub-band and a series of high frequency sub-bands. To improve the fusion performance, we designed two measures for fusion of the low frequency and the high frequency: the low frequency is divided into salient and nonsalient regions in accordance with the human visual system to improve the global contrast by targeted fusion and the high frequency requires a local contrast fusion strategy. Finally, the merged sub-bands are constructed according to the selection principles, and the final fused image is produced by applying the inverse NSST on these merged sub-bands. Experimental results demonstrate the effectiveness and superiority of the proposed method over the state-of-the-art fusion methods in terms of both visual effect and objective evaluation results.

[1]  E. Peli Contrast in complex images. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[2]  Ramachandra Raghavendra,et al.  Particle swarm optimization based fusion of near infrared and visible images for improved face verification , 2011, Pattern Recognit..

[3]  Baohua Zhang,et al.  A fusion algorithm for infrared and visible images based on saliency analysis and non-subsampled Shearlet transform , 2015 .

[4]  Soumika Munshi,et al.  Spectral band invariant wavelet-modified MACH filter , 2008 .

[5]  Biao Hou,et al.  SAR Image Despeckling Based on Nonsubsampled Shearlet Transform , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[7]  Wang-Q Lim,et al.  Sparse multidimensional representation using shearlets , 2005, SPIE Optics + Photonics.

[8]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[9]  Gitta Kutyniok,et al.  From Wavelets to Shearlets and back again , 2007 .

[10]  Gemma Piella,et al.  A general framework for multiresolution image fusion: from pixels to regions , 2003, Inf. Fusion.

[11]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[12]  Zhizhong Fu,et al.  Infrared and visible images fusion based on RPCA and NSCT , 2016 .

[13]  Xin Liu,et al.  A novel similarity based quality metric for image fusion , 2008, Inf. Fusion.

[14]  Wang Zhi-she,et al.  Multi-sensor image enhanced fusion algorithm based on NSST and top-hat transformation , 2015 .

[15]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[16]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[17]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[18]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[19]  K. Kannan,et al.  Optimal Decomposition Level of Discrete Wavelet Transform for Pixel Based Fusion of Multi - Focused Images , 2007, International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007).

[20]  Henry Selvaraj,et al.  Constrained Disjoint Paths in Geometric Network , 2007, International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007).

[21]  Qiang Fan,et al.  Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain , 2014 .

[22]  W. Kong Technique for gray-scale visual light and infrared image fusion based on non-subsampled shearlet transform , 2014 .

[23]  Yang Lei,et al.  Novel fusion method for visible light and infrared images based on NSST–SF–PCNN , 2014 .

[24]  Hua Zong,et al.  Infrared and visible image fusion based on visual saliency map and weighted least square optimization , 2017 .

[25]  Yang Lei,et al.  Visible and infrared image fusion using NSST and deep Boltzmann machine , 2018 .

[26]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[27]  Fang Liu,et al.  Fusion of infrared and visual images based on contrast pyramid directional filter banks using clonal selection optimizing , 2008 .

[28]  W. Kong,et al.  Technique for infrared and visible image fusion based on non-subsampled shearlet transform and spiking cortical model , 2015 .

[29]  Haiyan Jin,et al.  A fusion method for visible and infrared images based on contrast pyramid with teaching learning based optimization , 2014 .

[30]  Wei Huang,et al.  An improved fusion algorithm for infrared and visible images based on multi-scale transform , 2016 .

[31]  Ravindra Dhuli,et al.  Two-scale image fusion of visible and infrared images using saliency detection , 2016 .

[32]  Xiangzhi Bai,et al.  Morphological infrared image enhancement based on multi-scale sequential toggle operator using opening and closing as primitives , 2015 .

[33]  Yiling Yang,et al.  Thermal Physical Property-Based Fusion of Geostationary Meteorological Satellite Visible and Infrared Channel Images , 2014, Sensors.

[34]  Nong Sang,et al.  Attention-based hierarchical fusion of visible and infrared images , 2015 .

[35]  Fang Liu,et al.  Multicontourlet-Based Adaptive Fusion of Infrared and Visible Remote Sensing Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[36]  Quan Wang,et al.  Infrared and visible image fusion based on target extraction in the nonsubsampled contourlet transform domain , 2017 .

[37]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.