Context-Adaptive Pansharpening Based on Image Segmentation

Pansharpened images are widely used synthetic representations of the Earth surface characterized by both a high spatial resolution and a high spectral diversity. They are usually generated by extracting spatial details from a high-resolution PANchromatic image and by injecting them into a low spatial resolution multispectral image. The details injection is performed through injection coefficients, whose values can be either uniform for the whole image (global methods) or spatially variant (context-adaptive (CA) approaches). In this paper, we propose a CA approach in which the injection coefficients are estimated over image segments achieved through a binary partition tree segmentation algorithm. The approach is applied to two credited pansharpening algorithms based on the Gram–Schmidt orthogonalization procedure and the generalized Laplacian pyramid technique. The performance assessment is performed using two different data sets acquired by the QuickBird and the WorldView-3 satellites. The validation procedure, both at full and at reduced resolution, shows the suitability of the proposed approach, which reaches a good tradeoff between accuracy and computational burden.

[1]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[2]  Jocelyn Chanussot,et al.  A Pansharpening Method Based on the Sparse Representation of Injected Details , 2015, IEEE Geoscience and Remote Sensing Letters.

[3]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[4]  Johannes R. Sveinsson,et al.  A New Pansharpening Algorithm Based on Total Variation , 2014, IEEE Geoscience and Remote Sensing Letters.

[5]  Bruno Aiazzi,et al.  A Comparison Between Global and Context-Adaptive Pansharpening of Multispectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[6]  Jocelyn Chanussot,et al.  Improving MODIS Spatial Resolution for Snow Mapping Using Wavelet Fusion and ARSIS Concept , 2008, IEEE Geoscience and Remote Sensing Letters.

[7]  D. Roberts,et al.  Mapping forest degradation in the Eastern Amazon from SPOT 4 through spectral mixture models , 2003 .

[8]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[9]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Jocelyn Chanussot,et al.  Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique , 2008, IEEE Geoscience and Remote Sensing Letters.

[11]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Alan R. Gillespie,et al.  Color enhancement of highly correlated images. II. Channel ratio and “chromaticity” transformation techniques , 1987 .

[13]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[14]  A. Mohammadzadeh,et al.  Road extraction based on fuzzy logic and mathematical morphology from pan‐sharpened ikonos images , 2006 .

[15]  Luciano Alparone,et al.  Full scale assessment of pansharpening methods and data products , 2014, Remote Sensing.

[16]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Bin Luo,et al.  Pansharpening with a decision fusion based on the local size information , 2010, 2010 IEEE International Conference on Image Processing.

[19]  Aggelos K. Katsaggelos,et al.  A survey of classical methods and new trends in pansharpening of multispectral images , 2011, EURASIP J. Adv. Signal Process..

[20]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[21]  Andrea Garzelli,et al.  Pansharpening of Multispectral Images Based on Nonlocal Parameter Optimization , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[22]  L. Alparone,et al.  An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas , 2003, 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas.

[23]  Huixian Wang,et al.  A Robust Image Fusion Method Based on Local Spectral and Spatial Correlation , 2014, IEEE Geoscience and Remote Sensing Letters.

[24]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[26]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Antonio J. Plaza,et al.  Hyperspectral Image Segmentation Using a New Spectral Unmixing-Based Binary Partition Tree Representation , 2014, IEEE Transactions on Image Processing.

[28]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[29]  Te-Ming Tu,et al.  A new look at IHS-like image fusion methods , 2001, Inf. Fusion.

[30]  Francesca Bovolo,et al.  Analysis of the Effects of Pansharpening in Change Detection on VHR Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[31]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[32]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[33]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[35]  Bruno Aiazzi,et al.  Full-Scale Assessment of Pansharpening Through Polynomial Fitting of Multiscale Measurements , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Bin Luo,et al.  Decision-Based Fusion for Pansharpening of Remote Sensing Images , 2013, IEEE Geoscience and Remote Sensing Letters.

[38]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[39]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[40]  Lucien Wald,et al.  Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions , 2002 .

[41]  Thierry Ranchin,et al.  Liu 'Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details' , 2002 .

[42]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Jocelyn Chanussot,et al.  A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors , 2014, IEEE Transactions on Image Processing.

[44]  Jocelyn Chanussot,et al.  MultiResolution Analysis and Component Substitution techniques for hyperspectral Pansharpening , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[45]  Jocelyn Chanussot,et al.  Comparison of merging orders and pruning strategies for Binary Partition Tree in hyperspectral data , 2010, 2010 IEEE International Conference on Image Processing.

[46]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Jocelyn Chanussot,et al.  Context-adaptive Pansharpening based on binary partition tree segmentation , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[48]  Jocelyn Chanussot,et al.  Global and local Gram-Schmidt methods for hyperspectral pansharpening , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[49]  Jocelyn Chanussot,et al.  Pansharpening Quality Assessment Using the Modulation Transfer Functions of Instruments , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[50]  N. Keshava,et al.  Distance metrics and band selection in hyperspectral processing with applications to material identification and spectral libraries , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[51]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[52]  Philippe Salembier,et al.  Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval , 2000, IEEE Trans. Image Process..

[53]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[54]  Silvia Valero Valbuena Hyperspectral image representation and processing with binary partition trees , 2012 .

[55]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[56]  T. Ranchin,et al.  Liu 'Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details' , 2002 .

[57]  S. Baronti,et al.  Twenty-Five Years of Pansharpening: A Critical Review and New Developments , 2012 .