Texture-Guided Multisensor Superresolution for Remotely Sensed Images

This paper presents a novel technique, namely texture-guided multisensor superresolution (TGMS), for fusing a pair of multisensor multiresolution images to enhance the spatial resolution of a lower-resolution data source. TGMS is based on multiresolution analysis, taking object structures and image textures in the higher-resolution image into consideration. TGMS is designed to be robust against misregistration and the resolution ratio and applicable to a wide variety of multisensor superresolution problems in remote sensing. The proposed methodology is applied to six different types of multisensor superresolution, which fuse the following image pairs: multispectral and panchromatic images, hyperspectral and panchromatic images, hyperspectral and multispectral images, optical and synthetic aperture radar images, thermal-hyperspectral and RGB images, and digital elevation model and multispectral images. The experimental results demonstrate the effectiveness and high general versatility of TGMS.

[1]  Jocelyn Chanussot,et al.  A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors , 2014, IEEE Transactions on Image Processing.

[2]  Aleksandra Pizurica,et al.  Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Phaedon C. Kyriakidis,et al.  Fusion of MODIS Images Using Kriging With External Drift , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[5]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[6]  Eyal Ben Dor,et al.  SHALOM – A Commercial Hyperspectral Space Mission , 2015 .

[7]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[8]  Jianglin Ma,et al.  Superresolution Enhancement of Hyperspectral CHRIS/Proba Images With a Thin-Plate Spline Nonrigid Transform Model , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Johannes R. Sveinsson,et al.  Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Naoto Yokoya,et al.  Potential of Resolution-Enhanced Hyperspectral Data for Mineral Mapping Using Simulated EnMAP and Sentinel-2 Images , 2016, Remote. Sens..

[12]  Joëlle Thollot,et al.  Automatic texture guided color transfer and colorization , 2016 .

[13]  R. Green,et al.  NASA Mission to Measure Global Plant Physiology and Functional Types , 2008, 2008 IEEE Aerospace Conference.

[14]  Mario Chica-Olmo,et al.  Downscaling cokriging for image sharpening , 2006 .

[15]  Akira Iwasaki,et al.  Hyperspectral Imager Suite (HISUI) -Japanese hyper-multi spectral radiometer , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[16]  Stefania Matteoli,et al.  The PRISMA hyperspectral mission: Science activities and opportunities for agriculture and land monitoring , 2013, 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS.

[17]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[18]  S. Baronti,et al.  Twenty-Five Years of Pansharpening: A Critical Review and New Developments , 2012 .

[19]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[20]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[21]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[22]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Wenzhong Shi,et al.  Fusion of Sentinel-2 images , 2016 .

[24]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Aykut Erdem,et al.  Structure-preserving image smoothing via region covariances , 2013, ACM Trans. Graph..

[26]  Naoto Yokoya,et al.  Cross-Calibration for Data Fusion of EO-1/Hyperion and Terra/ASTER , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Qian Du,et al.  Hyperspectral and LiDAR Data Fusion: Outcome of the 2013 GRSS Data Fusion Contest , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  Bin Wang,et al.  Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods , 2014, IEEE Geoscience and Remote Sensing Letters.

[29]  Patrick Hostert,et al.  The EnMAP Spaceborne Imaging Spectroscopy Mission for Earth Observation , 2015, Remote. Sens..

[30]  Peter Reinartz,et al.  Mutual-Information-Based Registration of TerraSAR-X and Ikonos Imagery in Urban Areas , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Luciano Alparone,et al.  Hyperspectral pansharpening based on modulation of pixel spectra , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[32]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[33]  Peter Reinartz,et al.  ENHANCING URBAN DIGITAL ELEVATION MODELS USING AUTOMATED COMPUTER VISION TECHNIQUES , 2010 .

[34]  Philippe Gamet,et al.  HYPXIM — A hyperspectral satellite defined for science, security and defence users , 2011, 2011 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[35]  Bruno Aiazzi,et al.  Hyper-Sharpening: A First Approach on SIM-GA Data , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[36]  Jocelyn Chanussot,et al.  Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data , 2014, IEEE Transactions on Image Processing.

[37]  P. Atkinson,et al.  Downscaling MODIS images with area-to-point regression kriging , 2015 .

[38]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Claus Brenner,et al.  Extraction of buildings and trees in urban environments , 1999 .

[40]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[41]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[42]  M. S. Moran,et al.  A window-based technique for combining landsat thematic mapper thermal data with higher-resolution multispectral data over agricultural lands , 1990 .

[43]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[44]  David Krutz,et al.  DESIS (DLR Earth Sensing Imaging Spectrometer for the ISS-MUSES platform) , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[45]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[46]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[48]  Fred A. Kruse,et al.  The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data , 1993 .

[49]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[50]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Russell C. Hardie,et al.  Application of the stochastic mixing model to hyperspectral resolution enhancement , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[52]  M. F. Baumgardner,et al.  220 Band AVIRIS Hyperspectral Image Data Set: June 12, 1992 Indian Pine Test Site 3 , 2015 .