Multifocus image fusion by combining with mixed-order structure tensors and multiscale neighborhood

A new fusion scheme based on multiscale neighborhood technique is proposed for multifocus image fusion.A new focus measure by combining the structure tensors of mixed order differential is developed.An effective focused region detection method is presented based on multiscale neighborhood technique.An averaging fusion scheme is presented based on the fusion decision maps of different scale for uncertain regions.Our method outperforms the traditional methods in various experiments. In this study, we propose a new method for multifocus image fusion by combining with the structure tensors of mixed order differentials and the multiscale neighborhood. In this method, the structure tensor of an integral differential is utilized to detect the high frequency regions and the structure tensor of the fractional differential is used to detect the low frequency regions. To improve the performance of the fusion method, we propose a new focus measure based on the multiscale neighborhood technique to generate the initial fusion decision maps by exploiting the advantages of different scales. Next, based on the multiscale neighborhood technique, a post-processing method is used to update the initial fusion decision maps. During the fusion process, the pixels located in the focused inner regions are selected to produce the fused image. In order to avoid discontinuities in the transition zone between the focused and defocused regions, we propose a new "averaging" scheme based on the fusion decision maps at different scales. Our experimental results demonstrate that the proposed method outperformed the conventional multifocus image fusion methods in terms of both their subjective and objective quality.

[1]  Bo Du,et al.  Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding , 2015, Pattern Recognit..

[2]  Phen-Lan Lin,et al.  Fusion methods based on dynamic-segmented morphological wavelet or cut and paste for multifocus images , 2008, Signal Process..

[3]  Yuan Yan Tang,et al.  Multi-focus image fusion based on the neighbor distance , 2013, Pattern Recognit..

[4]  Sim Heng Ong,et al.  Autofocusing for tissue microscopy , 1993, Image Vis. Comput..

[5]  Henry Leung,et al.  Covariance intersection based image fusion technique with application to pansharpening in remote sensing , 2010, Inf. Sci..

[6]  Yi Chai,et al.  A new fusion scheme for multifocus images based on focused pixels detection , 2013, Machine Vision and Applications.

[7]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[8]  Dacheng Tao,et al.  On the Performance of Manhattan Nonnegative Matrix Factorization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Dacheng Tao,et al.  Classification with Noisy Labels by Importance Reweighting , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[11]  Yu Liu,et al.  Multi-focus image fusion with dense SIFT , 2015, Inf. Fusion.

[12]  Zheng Liu,et al.  A new contrast based multimodal medical image fusion framework , 2015, Neurocomputing.

[13]  Xin Liu,et al.  A novel similarity based quality metric for image fusion , 2008, Inf. Fusion.

[14]  Rick S. Blum,et al.  A new automated quality assessment algorithm for image fusion , 2009, Image Vis. Comput..

[15]  Ling Shao,et al.  Multiview Alignment Hashing for Efficient Image Search , 2015, IEEE Transactions on Image Processing.

[16]  Vassilia Karathanassi,et al.  Investigation of the Dual-Tree Complex and Shift-Invariant Discrete Wavelet Transforms on Quickbird Image Fusion , 2007, IEEE Geoscience and Remote Sensing Letters.

[17]  Xuelong Li,et al.  Principal Component 2-D Long Short-Term Memory for Font Recognition on Single Chinese Characters , 2016, IEEE Transactions on Cybernetics.

[18]  Xiangyang Wang,et al.  A new robust color image watermarking using local quaternion exponent moments , 2014, Inf. Sci..

[19]  Bo Du,et al.  A sparse and discriminative tensor to vector projection for human gait feature representation , 2015, Signal Process..

[20]  Shutao Li,et al.  Image matting for fusion of multi-focus images in dynamic scenes , 2013, Inf. Fusion.

[21]  Gaofeng Meng,et al.  Multifocus image fusion via focus segmentation and region reconstruction , 2014, Neurocomputing.

[22]  Rick S. Blum,et al.  A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application , 1999, Proc. IEEE.

[23]  C. L. Philip Chen,et al.  Robust Mel-Frequency Cepstral coefficients feature detection and dual-tree complex wavelet transform for digital audio watermarking , 2015, Inf. Sci..

[24]  Shadrokh Samavi,et al.  Multi-focus image fusion using dictionary-based sparse representation , 2015, Inf. Fusion.

[25]  Haitao Yin,et al.  Sparse representation with learned multiscale dictionary for image fusion , 2015, Neurocomputing.

[26]  Shree K. Nayar,et al.  Shape from Focus , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Gemma Piella,et al.  Image Fusion for Enhanced Visualization: A Variational Approach , 2009, International Journal of Computer Vision.

[28]  Liangpei Zhang,et al.  A Multifeature Tensor for Remote-Sensing Target Recognition , 2011, IEEE Geoscience and Remote Sensing Letters.

[29]  Zhongliang Jing,et al.  Evaluation of focus measures in multi-focus image fusion , 2007, Pattern Recognit. Lett..

[30]  Jingwen Yan,et al.  High quality multi-focus image fusion using self-similarity and depth information , 2015 .

[31]  Bhabatosh Chanda,et al.  Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure , 2013, Inf. Fusion.

[32]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[33]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[34]  Xiaohui Yuan,et al.  A Wavelet-Based Noise-Aware Method for Fusing Noisy Imagery , 2007, 2007 IEEE International Conference on Image Processing.

[35]  Gonzalo Pajares,et al.  A wavelet-based image fusion tutorial , 2004, Pattern Recognit..

[36]  Qiang Zhang,et al.  Multifocus image fusion using the nonsubsampled contourlet transform , 2009, Signal Process..

[37]  Henry Leung,et al.  A variational Bayesian approach to robust sensor fusion based on Student-t distribution , 2013, Inf. Sci..

[38]  Dacheng Tao,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Xuelong Li,et al.  Supervised Tensor Learning , 2005, ICDM.

[40]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[41]  Jimeng Sun,et al.  Beyond streams and graphs: dynamic tensor analysis , 2006, KDD '06.

[42]  Sun Li,et al.  Multi-scale weighted gradient-based fusion for multi-focus images , 2014, Inf. Fusion.

[43]  Zhengtao Yu,et al.  Fractional differential and variational method for image fusion and super-resolution , 2016, Neurocomputing.

[44]  L. Yang,et al.  Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform , 2008, Neurocomputing.

[45]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[46]  Vladimir S. Petrovic,et al.  Sensor noise effects on signal-level image fusion performance , 2003, Inf. Fusion.

[47]  Gabriel Cristóbal,et al.  Multifocus image fusion using the log-Gabor transform and a Multisize Windows technique , 2009, Inf. Fusion.

[48]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[49]  Ling Shao,et al.  Weakly-Supervised Cross-Domain Dictionary Learning for Visual Recognition , 2014, International Journal of Computer Vision.

[50]  K. K. Sharma,et al.  Hybrid image fusion scheme using self-fractional Fourier functions and multivariate empirical mode decomposition , 2014, Signal Process..

[51]  F. Tatom THE RELATIONSHIP BETWEEN FRACTIONAL CALCULUS AND FRACTALS , 1995 .

[52]  Victor Vianu,et al.  Invited articles section foreword , 2010, JACM.

[53]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[54]  Jianguo Jiang,et al.  Directional projection based image fusion quality metric , 2014, Inf. Sci..

[55]  Yi-Fei Pu,et al.  Fractional Differential Mask: A Fractional Differential-Based Approach for Multiscale Texture Enhancement , 2010, IEEE Transactions on Image Processing.

[56]  Yu Zhang,et al.  Image denoising using SVM classification in nonsubsampled contourlet transform domain , 2013, Inf. Sci..

[57]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[58]  Shutao Li,et al.  Multifocus image fusion by combining curvelet and wavelet transform , 2008, Pattern Recognit. Lett..

[59]  Danilo P. Mandic,et al.  Multiscale Image Fusion Using Complex Extensions of EMD , 2009, IEEE Transactions on Signal Processing.

[60]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[61]  Wenzhong Shi,et al.  Multisource Image Fusion Method Using Support Value Transform , 2007, IEEE Transactions on Image Processing.

[62]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[63]  Zheng Liu,et al.  PERFORMANCE ASSESSMENT OF COMBINATIVE PIXEL-LEVEL IMAGE FUSION BASED ON AN ABSOLUTE FEATURE MEASUREMENT , 2007 .

[64]  Hong Liu,et al.  Improved direction estimation for Di Zenzo's multichannel image gradient operator , 2012, Pattern Recognit..

[65]  Xuelong Li,et al.  Hyperspectral biological images compression based on multiway tensor projection , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[66]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[67]  Bruce J. West,et al.  Fractional Calculus and the Evolution of Fractal Phenomena , 1999 .

[68]  Yi Chai,et al.  Multifocus image fusion scheme using focused region detection and multiresolution , 2011 .

[69]  Mingli Song,et al.  Manifold Ranking-Based Matrix Factorization for Saliency Detection , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[70]  Gwanggil Jeon,et al.  A multifocus image fusion method by using hidden Markov model , 2013 .

[71]  Shutao Li,et al.  Multifocus image fusion using region segmentation and spatial frequency , 2008, Image Vis. Comput..

[72]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Derong Liu TNNLS Call for Reviewers and Special Issues , 2015, IEEE Trans. Neural Networks Learn. Syst..

[74]  Yu Han,et al.  A new image fusion performance metric based on visual information fidelity , 2013, Inf. Fusion.

[75]  Dacheng Tao,et al.  Multi-View Intact Space Learning , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.