Fractional differential and variational method for image fusion and super-resolution

This paper introduces a novel fractional differential and variational model that includes the terms of fusion and super-resolution, edge enhancement and noise suppression. In image fusion and super-resolution term, the structure tensor is employed to describe the geometry of all the input images. According to the fact that the fused image and the source inputs should have the same or similar structure tensor, the energy functional of the image fusion and super-resolution is established combining with the down-sampling operator. For edge enhancement, the bidirectional diffusion term is incorporated into the image fusion and super-resolution model to enhance the visualization of the fused image. In the noise suppression term, a new variational model is developed based on the fractional differential and fractional total variation. Thanks to the above three terms, the proposed model can realize the image fusion, super-resolution, and the edge information enhancement simultaneously. To search for the optimal solution, a gradient descent iteration scheme derived from the Euler-Lagrange equation of the proposed model is employed. The numerical results indicate that the proposed method is feasible and effective.

[1]  Stanley Osher,et al.  Image Super-Resolution by TV-Regularization and Bregman Iteration , 2008, J. Sci. Comput..

[2]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[3]  VetterliM.,et al.  The contourlet transform , 2005 .

[4]  Yi-Fei Pu,et al.  Fractional Differential Mask: A Fractional Differential-Based Approach for Multiscale Texture Enhancement , 2010, IEEE Transactions on Image Processing.

[5]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[6]  Cedric Nishan Canagarajah,et al.  Non-Gaussian model-based fusion of noisy images in the wavelet domain , 2010, Comput. Vis. Image Underst..

[7]  Qiuqi Ruan,et al.  A feature-dependent fuzzy bidirectional flow for adaptive image sharpening , 2007, Neurocomputing.

[8]  Qiang Zhang,et al.  Multifocus image fusion using the nonsubsampled contourlet transform , 2009, Signal Process..

[9]  Shutao Li,et al.  Multifocus image fusion by combining curvelet and wavelet transform , 2008, Pattern Recognit. Lett..

[10]  Yong Jiang,et al.  Image fusion with morphological component analysis , 2014, Inf. Fusion.

[11]  Shuai Ding,et al.  Image Fusion Algorithm Based on Nonsubsampled Contourlet Transform , 2013 .

[12]  Rui Li,et al.  Tunable-Q contourlet-based multi-sensor image fusion , 2013, Signal Process..

[13]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[14]  Haitao Yin,et al.  Sparse representation with learned multiscale dictionary for image fusion , 2015, Neurocomputing.

[15]  Gemma Piella,et al.  Image Fusion for Enhanced Visualization: A Variational Approach , 2009, International Journal of Computer Vision.

[16]  Olivier Lavialle,et al.  Multifocus image fusion and denoising: A variational approach , 2012, Pattern Recognit. Lett..

[17]  ZhouJi-Liu,et al.  Fractional differential mask , 2010 .

[18]  Yuan Yan Tang,et al.  Multi-focus image fusion based on the neighbor distance , 2013, Pattern Recognit..

[19]  Shutao Li,et al.  Simultaneous image fusion and super-resolution using sparse representation , 2013, Inf. Fusion.

[20]  Gonzalo Pajares Martinsanz,et al.  A wavelet-based image fusion tutorial , 2004 .

[21]  Lawrence B. Wolff,et al.  Multispectral image visualization through first-order fusion , 2002, IEEE Trans. Image Process..

[22]  Yi Chai,et al.  A new fusion scheme for multifocus images based on focused pixels detection , 2013, Machine Vision and Applications.

[23]  Rick S. Blum.  On multisensor image fusion performance limits from an estimation theory perspective , 2006, Inf. Fusion.

[24]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[25]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[26]  Hong Liu,et al.  Improved direction estimation for Di Zenzo's multichannel image gradient operator , 2012, Pattern Recognit..

[27]  Chuanjiang He,et al.  Fractional order total variation regularization for image super-resolution , 2013, Signal Process..

[28]  L. Yang,et al.  Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform , 2008, Neurocomputing.

[29]  Vladimir S. Petrovic,et al.  Sensor noise effects on signal-level image fusion performance , 2003, Inf. Fusion.

[30]  LudusanCosmin,et al.  Multifocus image fusion and denoising , 2012 .

[31]  Mrityunjay Kumar,et al.  A Total Variation-Based Algorithm for Pixel-Level Image Fusion , 2009, IEEE Transactions on Image Processing.

[32]  N. DoM.,et al.  The Nonsubsampled Contourlet Transform , 2006 .

[33]  Jian Bai,et al.  Fractional-Order Anisotropic Diffusion for Image Denoising , 2007, IEEE Transactions on Image Processing.

[34]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[35]  Mingliang Xu,et al.  High quality multi-spectral and panchromatic image fusion technologies based on Curvelet transform , 2015, Neurocomputing.

[36]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[37]  Aggelos K. Katsaggelos,et al.  Variational Bayesian Super Resolution , 2011, IEEE Transactions on Image Processing.

[38]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[39]  Xiaohui Yuan,et al.  A Wavelet-Based Noise-Aware Method for Fusing Noisy Imagery , 2007, 2007 IEEE International Conference on Image Processing.