The PAN and MS image fusion algorithm based on adaptive guided filtering and gradient information regulation

Abstract In recent years, with the improvement in the accuracy of remote sensing image classification and target recognition, the feature level fusion technology of remote sensing images has attracted much attention and become a research hotspot. However, this kind of fusion technology is not as mature as pixel-level fusion technology, and there are still many problems to be solved. This paper proposes a multi-spectral (MS) and panchromatic (PAN) image fusion algorithm based on adaptive textural feature extraction and information injection regulation. The fusion algorithm includes two stages. The first stage extracts the textural details of high-resolution PAN images. In this stage, based on the sensitivity of the remote sensing images to the gray-level co-occurrence matrix (GLCM), an adaptive guided filter (AGIF) scheme for remote sensing images based on the GLCM is proposed. The feature information of the textures and details of the PAN image was fully extracted. The second stage injects the extracted feature information of the PAN image into an MS image. In this stage, a decision map based on the MS image gradient domain and a weighted matrix based on the gradient entropy measure were proposed in order to, respectively, realize the adaptability of the feature injection location selection and regulate the intensity of the injected information to the MS image. This ensures the rationality of the injection of the textural information and avoids noise, patches and other information interference. The proposed algorithm has the advantages of fully extracting the textural features of high-resolution PAN images, adaptively adjusting the injection position and intensity when injecting the feature information into an MS image, and providing the fused image with clear features. On the premise of effectively maintaining the spectral information quality, the spatial resolution of the fused image is improved. A large number of simulation experiments verify the effectiveness of the proposed method.

[1]  Rick S. Blum,et al.  Fusing synergistic information from multi-sensor images: An overview from implementation to performance assessment , 2018, Inf. Fusion.

[2]  Yu Liu,et al.  Simultaneous image fusion and denoising with adaptive sparse representation , 2015, IET Image Process..

[3]  Xiaojun Wu,et al.  Image Fusion With Contextual Statistical Similarity and Nonsubsampled Shearlet Transform , 2017, IEEE Sensors Journal.

[4]  Shiqian Wu,et al.  Weighted Guided Image Filtering , 2016, IEEE Transactions on Image Processing.

[5]  Liangpei Zhang,et al.  A Remote Sensing Image Fusion Method Based on the Analysis Sparse Model , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  Josef Kittler,et al.  Infrared and Visible Image Fusion using a Deep Learning Framework , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[7]  David A. Clausi,et al.  Comparing cooccurrence probabilities and Markov random fields for texture analysis of SAR sea ice imagery , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Yalou Huang,et al.  Multisensor Information Fusion for People Tracking With a Mobile Robot: A Particle Filtering Approach , 2015, IEEE Transactions on Instrumentation and Measurement.

[9]  Gabriele Moser,et al.  Decision Fusion With Multiple Spatial Supports by Conditional Random Fields , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Mingjing Li,et al.  Review of image fusion algorithm based on multiscale decomposition , 2013, Proceedings 2013 International Conference on Mechatronic Sciences, Electric Engineering and Computer (MEC).

[11]  C. V. Jiji,et al.  A novel remote sensing image fusion algorithm using ICA bases , 2015, 2015 Eighth International Conference on Advances in Pattern Recognition (ICAPR).

[12]  Michael Meinild Nielsen,et al.  Remote sensing for urban planning and management: The use of window-independent context segmentation to extract urban features in Stockholm , 2015, Comput. Environ. Urban Syst..

[13]  Bin Luo,et al.  Decision-Based Fusion for Pansharpening of Remote Sensing Images , 2013, IEEE Geoscience and Remote Sensing Letters.

[14]  Hassan Ghassemian,et al.  A review of remote sensing image fusion methods , 2016, Inf. Fusion.

[15]  Xiao Xiang Zhu,et al.  Data Fusion and Remote Sensing: An ever-growing relationship , 2016, IEEE Geoscience and Remote Sensing Magazine.

[16]  Johannes R. Sveinsson,et al.  Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Qingshan Liu,et al.  Improving the Spatial Resolution of Landsat TM/ETM+ Through Fusion With SPOT5 Images via Learning-Based Super-Resolution , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[19]  P. Milanfar,et al.  Multiscale principal components analysis for image local orientation estimation , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[20]  Vijay Solanky,et al.  Pixel-level image fusion techniques in remote sensing: a review , 2016, Spatial Information Research.

[21]  Zhengguo Li,et al.  Gradient Domain Guided Image Filtering , 2015, IEEE Transactions on Image Processing.

[22]  Xiangyu Liu,et al.  Remote Sensing Image Fusion Based on Two-Stream Fusion Network , 2017, MMM.

[23]  Farhad Samadzadegan,et al.  Object Recognition Based on the Context Aware Decision-Level Fusion in Multiviews Imagery , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[24]  James M. Conrad,et al.  A survey of multisensor fusion techniques, architectures and methodologies , 2017, SoutheastCon 2017.

[25]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  H. Ghassemian,et al.  Improving hyperspectral image classification by combining spectral, texture, and shape features , 2015 .

[27]  Abhishek Dey,et al.  Remote sensing image fusion using Statistical Univariate Finite Mixture Model in Shearlet Domain , 2015, 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[28]  Jun Zhou,et al.  Object Classification via Feature Fusion Based Marginalized Kernels , 2015, IEEE Geoscience and Remote Sensing Letters.

[29]  Wenda Zhao,et al.  Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion , 2016, Inf. Fusion.

[30]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[32]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[33]  Xiaosong Li,et al.  Multifocus image fusion by combining with mixed-order structure tensors and multiscale neighborhood , 2016, Inf. Sci..

[34]  P. Maillard,et al.  SPECTRAL-TEXTURAL IMAGE CLASSIFICATION IN A SEMIARID ENVIRONMENT , 2006 .

[35]  Leen-Kiat Soh,et al.  Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices , 1999, IEEE Trans. Geosci. Remote. Sens..