Infrared and visible images fusion by using sparse representation and guided filter

Abstract Infrared and visible images play an important role in transportation systems since they can monitor traffic conditions around the clock. However, visible images are susceptible to the imaging environments, and infrared images are not rich enough in detail. The infrared and visible images fusion techniques can fuse these two different modal images into a single image with more useful information. In this paper, we propose an effective infrared and visible images fusion method for traffic systems. The weight maps are measured by utilizing the sparse coefficients. The next is to decompose the infrared and visible pair into high-frequency layers (HFLs) and low-frequency layers (LFLs). Since the two layers contain different structures and texture information, to extract the representative component, the guided filter is utilized to optimize weight maps in accordance with the different characteristic of the infrared and visible pairs. The final step is to reconstruct the two-scale layers according to the weight maps. Experimental results demonstrate our method outperforms other popular approaches in terms of subjective perception and objective metrics.

[1]  Hadi Seyedarabi,et al.  A non-reference image fusion metric based on mutual information of image features , 2011, Comput. Electr. Eng..

[2]  Shadrokh Samavi,et al.  Multi-focus image fusion using dictionary-based sparse representation , 2015, Inf. Fusion.

[3]  Gwanggil Jeon,et al.  Multi-Focus Image Fusion Method for Vision Sensor Systems via Dictionary Learning with Guided Filter , 2018, Sensors.

[4]  LiuKai,et al.  A new framework for remote sensing image super-resolution , 2016 .

[5]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[6]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Kai Liu,et al.  A new framework for remote sensing image super-resolution: Sparse representation-based method by processing dictionaries with multi-type features , 2016, J. Syst. Archit..

[8]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[9]  Qiguang Miao,et al.  A novel adaptive multi-focus image fusion algorithm based on PCNN and sharpness , 2005, SPIE Defense + Commercial Sensing.

[10]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[11]  Yi Zhou,et al.  Pixel-Level Fusion for Infrared and Visible Acquisitions , 2009 .

[12]  Yongjun Zhang,et al.  Salient Object Detection via Recursive Sparse Representation , 2018, Remote. Sens..

[13]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[14]  Kai Liu,et al.  Multiple dictionary pairs learning and sparse representation-based infrared image super-resolution with improved fuzzy clustering , 2018, Soft Comput..

[15]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[16]  Yu Liu,et al.  Infrared and visible image fusion with convolutional neural networks , 2017, Int. J. Wavelets Multiresolution Inf. Process..

[17]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[19]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[20]  Dahui Li,et al.  Effective detection by fusing visible and infrared images of targets for Unmanned Surface Vehicles , 2018, Automatika.

[21]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[22]  Sun Li,et al.  Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters , 2016, Inf. Fusion.

[23]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[25]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[26]  Qiang Zhang,et al.  Multifocus image fusion using the nonsubsampled contourlet transform , 2009, Signal Process..

[27]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Dan Roth,et al.  Learning to detect objects in images via a sparse, part-based representation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Junjun Jiang,et al.  FusionGAN: A generative adversarial network for infrared and visible image fusion , 2019, Inf. Fusion.

[30]  Shudao Zhou,et al.  Image fusion based on wavelet transform and gray-level features , 2019 .

[31]  Wei Wu,et al.  Pansharpening multispectral remote‐sensing images with guided filter for monitoring impact of human behavior on environment , 2018, Concurr. Comput. Pract. Exp..

[32]  Vladimir S. Petrovic,et al.  Gradient-based multiresolution image fusion , 2004, IEEE Transactions on Image Processing.

[33]  Yan Li,et al.  Direct generation of level of service maps from images using convolutional and long short-term memory networks , 2019, J. Intell. Transp. Syst..

[34]  Luciano Alparone,et al.  Remote sensing image fusion using the curvelet transform , 2007, Inf. Fusion.

[35]  Yu Liu,et al.  Simultaneous image fusion and denoising with adaptive sparse representation , 2015, IET Image Process..

[36]  Dan Roth,et al.  Learning a Sparse Representation for Object Detection , 2002, ECCV.

[37]  Alexander Toet,et al.  Image fusion by a ration of low-pass pyramid , 1989, Pattern Recognit. Lett..

[38]  Yu Liu,et al.  Multi-focus image fusion with a deep convolutional neural network , 2017, Inf. Fusion.

[39]  Keemin Sohn,et al.  Artificial intelligence for traffic signal control based solely on video images , 2018, J. Intell. Transp. Syst..

[40]  Ping Zhang,et al.  Multi-sensor image super-resolution with fuzzy cluster by using multi-scale and multi-view sparse coding for infrared image , 2017, Multimedia Tools and Applications.