Visible and infrared image fusion using NSST and deep Boltzmann machine

Abstract This paper proposes a novel fusion method for visible and infrared images based on non-subsampled shearlet transform (NSST) and deep boltzmann machine (DBM). As a typical model in the area of deep learning, DBM has remarked superiorities over several current models in terms of the function efficiency and final results. On the other hand, NSST is a novel multi-scale geometry analysis tool, and recent experimental results show that it has not only much better feature capturing ability, but also much lower computational complexities. In this paper, NSST is responsible for decomposing the source images into a series of sub-images and reconstructing the final fused image. DBM is used for conduct the coefficients selection in the sub-images. The simulation experimental results show that the proposed method has obvious advantages over the current fusion methods.

[1]  Wang Zhi-she,et al.  Multi-sensor image enhanced fusion algorithm based on NSST and top-hat transformation , 2015 .

[2]  Yang Lei,et al.  Fusion technique for multi-focus images based on NSCT–ISCM , 2015 .

[3]  W. Kong,et al.  Technique for image fusion between gray-scale visual light and infrared images based on NSST and improved RF , 2013 .

[4]  Pan Lin,et al.  Multimodal Sensor Medical Image Fusion Based on Type-2 Fuzzy Logic in NSCT Domain , 2016, IEEE Sensors Journal.

[5]  Yang Lei,et al.  Novel fusion method for visible light and infrared images based on NSST–SF–PCNN , 2014 .

[6]  Tapani Raiko,et al.  Deep Learning Made Easier by Linear Transformations in Perceptrons , 2012, AISTATS.

[7]  Guo Bao.  FUSION OF INFRARED AND VISIBLE LIGHT IMAGES BASED ON NONSUBSAMPLED CONTOURLET TRANSFORM , 2007 .

[8]  Sun Xintong.  Fusion of visual and infrared images based on IHS and wavelet transforms , 2012 .

[9]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[10]  Wang Ke.  Color Image Fusion Algorithm Using the Contourlet Transform , 2007 .

[11]  W. Kong,et al.  Multi-sensor image fusion based on NSST domain I2CM , 2013 .

[12]  Li Yan,et al.  A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain , 2015 .

[13]  G. Easley,et al.  Sparse directional image representations using the discrete shearlet transform , 2008 .

[14]  W. Kong,et al.  Technique for image fusion based on NSST domain improved fast non-classical RF , 2013 .

[15]  Lei Yingjie.  Technique for image fusion based on NSST domain and human visual characteristics , 2013 .

[16]  Xavier Maldague,et al.  An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing , 2016 .

[17]  William A. Sethares,et al.  Paper texture classification via multi-scale Restricted Boltzman Machines , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[18]  Zhang Baohua,et al.  A multi-focus image fusion algorithm based on an improved dual-channel PCNN in NSCT domain , 2013 .

[19]  Yafei Zhang,et al.  Infrared and visible image fusion scheme based on NSCT and low-level visual features , 2016 .

[20]  Christian Wolf,et al.  Sequential Deep Learning for Human Action Recognition , 2011, HBU.

[21]  Pan Lin,et al.  Multifocus Image Fusion Based on NSCT and Focused Area Detection , 2014, IEEE Sensors Journal.

[22]  W. Kong,et al.  Technique for image fusion based on NSST domain INMF , 2014 .

[23]  Chunhui Zhao,et al.  A fast fusion scheme for infrared and visible light images in NSCT domain , 2015 .

[24]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[25]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.

[26]  Xu Li,et al.  A fusion algorithm for infrared and visible images based on RDU-PCNN and ICA-bases in NSST domain , 2016 .