Multi-focus image fusion using Content Adaptive Blurring

Abstract Multi-focus image fusion has emerged as an important research area in information fusion. It aims at increasing the depth-of-field by extracting focused regions from multiple partially focused images, and merging them together to produce a composite image in which all objects are in focus. In this paper, a novel multi-focus image fusion algorithm is presented in which the task of detecting the focused regions is achieved using a Content Adaptive Blurring (CAB) algorithm. The proposed algorithm induces non-uniform blur in a multi-focus image depending on its underlying content. In particular, it analyzes the local image quality in a neighborhood and determines if the blur should be induced or not without losing image quality. In CAB, pixels belonging to the blur regions receive little or no blur at all, whereas the focused regions receive significant blur. Absolute difference of the original image and the CAB-blurred image yields initial segmentation map, which is further refined using morphological operators and graph-cut techniques to improve the segmentation accuracy. Quantitative and qualitative evaluations and comparisons with current state-of-the-art on two publicly available datasets demonstrate the strength of the proposed algorithm.

[1]  Jing Tian,et al.  Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure , 2012, Signal Process..

[2]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[3]  Miao Qi-guang,et al.  A Novel Image Fusion Method Using Contourlet Transform , 2006, 2006 International Conference on Communications, Circuits and Systems.

[4]  Shadrokh Samavi,et al.  Multi-focus image fusion using dictionary-based sparse representation , 2015, Inf. Fusion.

[5]  Q Guihong,et al.  Medical image fusion by wavelet transform modulus maxima. , 2001, Optics express.

[6]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[7]  J. Campbell Introduction to remote sensing , 1987 .

[8]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[9]  Qiang Zhang,et al.  Multifocus image fusion using the nonsubsampled contourlet transform , 2009, Signal Process..

[10]  Qingquan Li,et al.  A comparative analysis of image fusion methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Pan Lin,et al.  Dual-Tree Complex Wavelet Transform and Image Block Residual-Based Multi-Focus Image Fusion in Visual Sensor Networks , 2014, Sensors.

[12]  Yi Shen,et al.  19 – Performance evaluation of image fusion techniques , 2008 .

[13]  Sabine Süsstrunk,et al.  Non-Parametric Blur Map Regression for Depth of Field Extension , 2016, IEEE Transactions on Image Processing.

[14]  Qiaoqiao Li,et al.  A Novel Explicit Multi-focus Image Fusion Method , 2015, J. Inf. Hiding Multim. Signal Process..

[15]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[16]  Chun-Jen Chen,et al.  A linear-time component-labeling algorithm using contour tracing technique , 2004, Comput. Vis. Image Underst..

[17]  屈小波 Xiaobo Qu,et al.  Image Fusion Algorithm Based on Spatial Frequency-Motivated Pulse Coupled Neural Networks in Nonsubsampled Contourlet Transform Domain , 2008 .

[18]  David Zhang,et al.  A survey of graph theoretical approaches to image segmentation , 2013, Pattern Recognit..

[19]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[20]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[21]  Lorenzo Bruzzone,et al.  Image fusion techniques for remote sensing applications , 2002, Inf. Fusion.

[22]  Shutao Li,et al.  Fast multi-exposure image fusion with median filter and recursive filter , 2012, IEEE Transactions on Consumer Electronics.

[23]  Hadi Seyedarabi,et al.  A non-reference image fusion metric based on mutual information of image features , 2011, Comput. Electr. Eng..

[24]  Kenji Suzuki,et al.  Linear-time connected-component labeling based on sequential local operations , 2003, Comput. Vis. Image Underst..

[25]  Shutao Li,et al.  Multifocus image fusion by combining curvelet and wavelet transform , 2008, Pattern Recognit. Lett..

[26]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[27]  B. K. Shreyamsha Kumar,et al.  Image fusion based on pixel significance using cross bilateral filter , 2013, Signal, Image and Video Processing.

[28]  Luciano Alparone,et al.  Remote sensing image fusion using the curvelet transform , 2007, Inf. Fusion.

[29]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[30]  Wei Cai,et al.  A region-based multi-sensor image fusion scheme using pulse-coupled neural network , 2006, Pattern Recognit. Lett..

[31]  Shutao Li,et al.  Image matting for fusion of multi-focus images in dynamic scenes , 2013, Inf. Fusion.

[32]  Yide Ma,et al.  Medical image fusion using m-PCNN , 2008, Inf. Fusion.

[33]  Nikolaos Mitianoudis,et al.  Pixel-based and region-based image fusion schemes using ICA bases , 2007, Inf. Fusion.

[34]  Sun Li,et al.  Multi-scale weighted gradient-based fusion for multi-focus images , 2014, Inf. Fusion.

[35]  Wei Liu,et al.  A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation , 2017, Neurocomputing.

[36]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  M. Hossny,et al.  Comments on 'Information measure for performance of image fusion' , 2008 .

[38]  Jingwen Yan,et al.  Image fusion algorithm based on orientation information motivated Pulse Coupled Neural Networks , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[39]  David Bull,et al.  Region-Based Multimodal Image Fusion Using ICA Bases , 2007 .

[40]  Mohammad Haghighat,et al.  Fast-FMI: Non-reference image fusion metric , 2014, 2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT).

[41]  Bhabatosh Chanda,et al.  Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure , 2013, Inf. Fusion.

[42]  B. K. Shreyamsha Kumar,et al.  Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform , 2013, Signal Image Video Process..

[43]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[44]  Aleksandra Pizurica,et al.  Extending the Depth of Field in Microscopy Through Curvelet-Based Frequency-Adaptive Image Fusion , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[45]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[47]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[48]  L. Yang,et al.  Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform , 2008, Neurocomputing.

[49]  Xin Liu,et al.  A novel similarity based quality metric for image fusion , 2008, Inf. Fusion.

[50]  Rick S. Blum,et al.  Multi-sensor image fusion and its applications , 2005 .

[51]  Q. Wang,et al.  A nonlinear correlation measure for multivariable data set , 2005 .

[52]  Hassan Ghassemian,et al.  A review of remote sensing image fusion methods , 2016, Inf. Fusion.

[53]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[54]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[55]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[56]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[57]  J. R. Raol,et al.  Pixel-level Image Fusion using Wavelets and Principal Component Analysis , 2008 .

[58]  Gonzalo Pajares,et al.  A wavelet-based image fusion tutorial , 2004, Pattern Recognit..

[59]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[60]  Y. Asnath Victy Phamila,et al.  Discrete Cosine Transform based fusion of multi-focus images for visual sensor networks , 2014, Signal Process..

[61]  Liu Cao,et al.  Multi-Focus Image Fusion Based on Spatial Frequency in Discrete Cosine Transform Domain , 2015, IEEE Signal Processing Letters.

[62]  Yun Zhang,et al.  Wavelet based image fusion techniques — An introduction, review and comparison , 2007 .

[63]  Vladimir S. Petrovic,et al.  Gradient-based multiresolution image fusion , 2004, IEEE Transactions on Image Processing.

[64]  Shutao Li,et al.  Multifocus image fusion using region segmentation and spatial frequency , 2008, Image Vis. Comput..

[65]  Jinbo Li,et al.  Regional multifocus image fusion using sparse representation. , 2013, Optics express.

[66]  Panajotis Agathoklis,et al.  Multi-Exposure and Multi-Focus Image Fusion in Gradient Domain , 2016, J. Circuits Syst. Comput..

[67]  Marco Grangetto,et al.  DOST: a distributed object segmentation tool , 2017, Multimedia Tools and Applications.

[68]  Stavri G. Nikolov,et al.  Image fusion: Advances in the state of the art , 2007, Inf. Fusion.

[69]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[70]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[71]  Hadi Seyedarabi,et al.  Multi-focus image fusion for visual sensor networks in DCT domain , 2011, Comput. Electr. Eng..

[72]  Zheng Liu,et al.  Image fusion by using steerable pyramid , 2001, Pattern Recognit. Lett..

[73]  Gabriel Cristóbal,et al.  Multifocus image fusion using the log-Gabor transform and a Multisize Windows technique , 2009, Inf. Fusion.

[74]  Yu Liu,et al.  Multi-focus image fusion with dense SIFT , 2015, Inf. Fusion.