Multi-scale Guided Image and Video Fusion: A Fast and Efficient Approach

AbstractIn this paper, we propose a general purpose, simple and fast fusion algorithm based on guided image filter. The proposed method can well combine useful source image information into the fused image supported by multi-scale image decomposition, structure transferring property, visual saliency detection, and weight map construction. Multi-scale image decomposition is appropriate to represent and manipulate image features at various scales. Structure transferring property enabled by our algorithm can induce structures of one source image into the other. A new visual saliency detection based on guided image filter introduced in this paper is able to extract significant regions from visually different images of the same scene. The choice of weight maps helped to integrate the complementary information pixel by pixel at each scale. Experimental outcomes of the proposed method are compared and analyzed with traditional and recent guided image filter-based fusion algorithms in terms of visual quality, fusion metrics and run time. In addition, to enhance fusion results further we made an effort to find a suitable image and video enhancement algorithm. The fusion performance analysis clearly indicates that the proposed method is very promising along with less run time.

[1]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[2]  Gang Xiao,et al.  Fusion of MRI and CT images using guided image filter and image statistics , 2017, Int. J. Imaging Syst. Technol..

[3]  Ravindra Dhuli,et al.  Two-scale image fusion of visible and infrared images using saliency detection , 2016 .

[4]  Kin-Man Lam,et al.  Saliency analysis based on multi-scale wavelet decomposition , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[5]  Sumit Budhiraja,et al.  Multimodal medical image fusion based on guided filtered multi-scale decomposition , 2016 .

[6]  Xiaohai He,et al.  Infrared and visible image fusion with the use of multi-scale edge-preserving decomposition and guided image filter , 2015 .

[7]  Abdul Ghafoor,et al.  All in focus fusion using guided filter , 2015, Multidimens. Syst. Signal Process..

[8]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.

[9]  S. M. Mahbubur Rahman,et al.  Bayesian Fusion of Ensemble of Multifocused Noisy Images , 2015, Circuits Syst. Signal Process..

[10]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Abd. Rahman Ramli,et al.  Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation , 2003, IEEE Trans. Consumer Electron..

[12]  Alexander Toet,et al.  Iterative guided image fusion , 2016, PeerJ Comput. Sci..

[13]  Vinay Kumar,et al.  A Novel Approach for Detail-Enhanced Exposure Fusion Using Guided Filter , 2014, TheScientificWorldJournal.

[14]  Zhao Jie,et al.  Medical Image Fusion Based on Rolling Guidance Filter and Spiking Cortical Model , 2015, Computational and mathematical methods in medicine.

[15]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[16]  Jui-Chiu Chiang,et al.  High-Dynamic-Range Image Generation and Coding for Multi-exposure Multi-view Images , 2016, Circuits, Systems, and Signal Processing.

[17]  Abdul Ghafoor,et al.  Wavelet and guided filter based multifocus fusion for noisy images , 2015 .

[18]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[19]  Rick S. Blum,et al.  Multi-sensor image fusion and its applications , 2005 .

[20]  Chao Zhang,et al.  Color Enhancement in Image Fusion , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[21]  Uday B. Desai,et al.  Multimodal image/video fusion rule using generalized pixel significance based on statistical properties of the neighborhood , 2013, Signal, Image and Video Processing.

[22]  Durga Prasad Bavirisetti,et al.  Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve Transform , 2016, IEEE Sensors Journal.

[23]  Yu Han,et al.  A new image fusion performance metric based on visual information fidelity , 2013, Inf. Fusion.

[24]  Ioannis Pavlidis,et al.  Infrared and visible image fusion for face recognition , 2004, SPIE Defense + Commercial Sensing.

[25]  Abdul Ghafoor,et al.  Improved Guided Image Fusion for Magnetic Resonance and Computed Tomography Imaging , 2014, TheScientificWorldJournal.

[26]  Min Gyo Chung,et al.  Recursively separated and weighted histogram equalization for brightness preservation and contrast enhancement , 2008, IEEE Transactions on Consumer Electronics.

[27]  Shuaiqi Liu,et al.  Image fusion based on complex-shearlet domain with guided filtering , 2015, Multidimensional Systems and Signal Processing.

[28]  Minfang Peng,et al.  Fault Diagnosis of Analog Circuits Using Systematic Tests Based on Data Fusion , 2013, Circuits Syst. Signal Process..

[29]  Gang Xiao,et al.  Relative Object Tracking Algorithm Based on Convolutional Neural Network for Visible and Infrared Video Sequences , 2018, ICVR 2018.

[30]  Klaus Steinnocher,et al.  Influence of image fusion approaches on classification accuracy: a case study , 2006 .

[31]  Yu-Chiang Frank Wang,et al.  Exploring Visual and Motion Saliency for Automatic Video Object Extraction , 2013, IEEE Transactions on Image Processing.

[32]  Yeong-Taeg Kim,et al.  Contrast enhancement using brightness preserving bi-histogram equalization , 1997 .

[33]  Uday B. Desai,et al.  Multifocus and multispectral image fusion based on pixel significance using multiresolution decomposition , 2013, Signal Image Video Process..

[34]  Zhifeng Gao,et al.  Fusion of infrared and visible images for night-vision context enhancement. , 2016, Applied optics.

[35]  Gang Xiao,et al.  A New Image and Video Fusion Method Based on Cross Bilateral Filter , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[36]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[37]  Akira Taguchi,et al.  Modified histogram equalization for image contrast enhancement , 2010, 2010 4th International Symposium on Communications, Control and Signal Processing (ISCCSP).

[38]  Durga Prasad Bavirisetti,et al.  Multi-filtering based edge preserving image fusion technique , 2017, Int. J. Serv. Technol. Manag..

[39]  Gang Xiao,et al.  Object Tracking via Deep Multi-View Compressive Model for Visible and Infrared Sequences , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[40]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Abdul Ghafoor,et al.  Weighted fusion of MRI and PET images based on fractal dimension , 2017, Multidimens. Syst. Signal Process..

[42]  A. S. Madhukumar,et al.  A Cooperative Spectrum Sensing Technique with Dynamic Frequency Boundary Detection and Information-Entropy-Fusion for Primary User Detection , 2011, Circuits Syst. Signal Process..

[43]  Alexander Toet,et al.  Multiscale image fusion through guided filtering , 2016, Security + Defence.

[44]  Qian Wang,et al.  A Novel Method of Signal Fusion Based on Dimension Expansion , 2018, Circuits Syst. Signal Process..

[45]  Dae San Kim,et al.  Home network message specification for white goods and its applications , 2002, IEEE Trans. Consumer Electron..