Infrared and visible image fusion using total variation model

Abstract Image fusion is a process of combining complementary information from multiple images of the same scene into an image, so that the resultant image contains a more accurate description of the scene than any of the individual source images. In this paper, we propose a novel fusion strategy for infrared (IR) and visible images based on total variation (TV) minimization. By constraining the fused image to have similar pixel intensities with the IR image and similar gradients with the visible image, it tends to simultaneously keep the thermal radiation and appearance information in the source images. We evaluate our method on a publicly available database with comparisons to other seven fusion methods. Our results have a major difference that the fused images look like sharpened IR images with detailed appearance information. The quantitative results demonstrate that our method also can achieve comparable metric values with other state-of-the-art methods.

[1]  Ruimin Hu,et al.  Noise robust face hallucination employing Gaussian-Laplacian mixture model , 2014, Neurocomputing.

[2]  Shutao Li,et al.  Image matting for fusion of multi-focus images in dynamic scenes , 2013, Inf. Fusion.

[3]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[4]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Hadi Seyedarabi,et al.  A non-reference image fusion metric based on mutual information of image features , 2011, Comput. Electr. Eng..

[6]  Gemma Piella,et al.  A general framework for multiresolution image fusion: from pixels to regions , 2003, Inf. Fusion.

[7]  Mingliang Xu,et al.  High quality multi-spectral and panchromatic image fusion technologies based on Curvelet transform , 2015, Neurocomputing.

[8]  Gonzalo Pajares,et al.  A wavelet-based image fusion tutorial , 2004, Pattern Recognit..

[9]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[10]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[11]  Liangpei Zhang,et al.  Regional Spatially Adaptive Total Variation Super-Resolution With Spatial Information Filtering and Clustering , 2013, IEEE Transactions on Image Processing.

[12]  Alexander Toet,et al.  Fusion of visible and thermal imagery improves situational awareness , 1997 .

[13]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[14]  Laure J. Chipman,et al.  Wavelets and image fusion , 1995, Proceedings., International Conference on Image Processing.

[15]  Bo Wang,et al.  Co-transduction for Shape Retrieval , 2010, ECCV.

[16]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[17]  Shutao Li,et al.  Pixel-level image fusion with simultaneous orthogonal matching pursuit , 2012, Inf. Fusion.

[18]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[19]  Alan L. Yuille,et al.  Non-Rigid Point Set Registration by Preserving Global and Local Structures , 2016, IEEE Transactions on Image Processing.

[20]  Wei Liu,et al.  Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Vladimir S. Petrovic,et al.  Gradient-based multiresolution image fusion , 2004, IEEE Transactions on Image Processing.

[22]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  N. DoM.,et al.  The Nonsubsampled Contourlet Transform , 2006 .

[24]  Ruimin Hu,et al.  Efficient single image super-resolution via graph-constrained least squares regression , 2013, Multimedia Tools and Applications.

[25]  Yide Ma,et al.  Spiking cortical model for multifocus image fusion , 2014, Neurocomputing.

[26]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[27]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[28]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[29]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[30]  Jing Li,et al.  A Simplified Method of Making Flexible Blue LEDs on a Plastic Substrate , 2015, IEEE Photonics Journal.

[31]  Xiang Ma,et al.  Sparse Support Regression for Image Super-Resolution , 2015, IEEE Photonics Journal.

[32]  Mrityunjay Kumar,et al.  A Total Variation-Based Algorithm for Pixel-Level Image Fusion , 2009, IEEE Transactions on Image Processing.

[33]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[34]  Simon J. Godsill,et al.  A Nonreference Image Fusion Metric Based on the Regional Importance Measure , 2009, IEEE Journal of Selected Topics in Signal Processing.

[35]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[36]  Vps Naidu,et al.  Image Fusion Technique using Multi-resolution Singular Value Decomposition , 2011 .

[37]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[38]  Wei Liu,et al.  SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified Framework , 2015, IEEE Transactions on Image Processing.

[39]  Guy Gilboa,et al.  A Total Variation Spectral Framework for Scale and Texture Analysis , 2014, SIAM J. Imaging Sci..

[40]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[41]  Andy M. Yip,et al.  Recent Developments in Total Variation Image Restoration , 2004 .

[42]  Fakhri Karray,et al.  Multisensor data fusion: A review of the state-of-the-art , 2013, Inf. Fusion.

[43]  Zheng Liu,et al.  A new contrast based multimodal medical image fusion framework , 2015, Neurocomputing.

[44]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[45]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[46]  Zhuowen Tu,et al.  Robust $L_{2}E$ Estimation of Transformation for Non-Rigid Registration , 2015, IEEE Transactions on Signal Processing.

[47]  Alexander Toet,et al.  Merging thermal and visual images by a contrast pyramid , 1989 .

[48]  Karim Faez,et al.  Infrared and visible image fusion using fuzzy logic and population-based optimization , 2012, Appl. Soft Comput..

[49]  Laure J. Chipman,et al.  Wavelets and image fusion , 1995, Optics + Photonics.

[50]  Luciano Alparone,et al.  Remote sensing image fusion using the curvelet transform , 2007, Inf. Fusion.

[51]  Longin Jan Latecki,et al.  Path Similarity Skeleton Graph Matching , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Alexander Toet,et al.  Image fusion by a ration of low-pass pyramid , 1989, Pattern Recognit. Lett..

[53]  Curtis R. Vogel,et al.  Iterative Methods for Total Variation Denoising , 1996, SIAM J. Sci. Comput..

[54]  Yihua Tan,et al.  Unsupervised Multilayer Feature Learning for Satellite Image Scene Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[55]  Seong G. Kong,et al.  Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition , 2007, International Journal of Computer Vision.

[56]  Xu Zhang,et al.  Image fusion with saliency map and interest points , 2016, Neurocomputing.

[57]  James Llinas,et al.  Multisensor Data Fusion , 1990 .

[58]  Minghui Zhang,et al.  A generalized relative total variation method for image smoothing , 2015, Multimedia Tools and Applications.