Pre-Processing of Panchromatic Images to Improve Object Detection in Pansharpened Images

In recent years, many techniques of fusion of multi-sensors satellite images have been developed. This article focuses on examining and improvement the usability of pansharpened images for object detection, especially when fusing data with a high GSD ratio. A methodology to improve an interpretative ability of pansharpening results is based on pre-processing of the panchromatic image using Logarithmic-Laplace filtration. The proposed approach was used to examine several different pansharpening methods and data sets with different spatial resolution ratios, i.e., from 1:4 to 1:60. The obtained results showed that the proposed approach significantly improves an object detection of fused images, especially for imagery data with a high-resolution ratio. The interpretative ability was assessed using qualitative method (based on image segmentation) and quantitative method (using an indicator based on the Speeded Up Robust Features (SURF) detector). In the case of combining data acquired with the same sensor the interpretative potential had improved by a dozen or so per cent. However, for data with a high resolution ratio, the improvement was several dozen, or even several hundred per cents, in the case of images blurred after pansharpening by the classic method (with original panchromatic image). Image segmentation showed that it is possible to recognize narrow objects that were originally blurred and difficult to identify. In addition, for panchromatic images acquired by WorldView-2, the proposed approach improved not only object detection but also the spectral quality of the fused image.

[1]  Jaewan Choi,et al.  Pansharpening Using Guided Filtering to Improve the Spatial Clarity of VHR Satellite Imagery , 2019, Remote. Sens..

[2]  M. Noviello,et al.  Combined application of pansharpening and enhancement methods to improve archaeological cropmark visibility and identification in QuickBird imagery: two case studies from Apulia, Southern Italy , 2013 .

[3]  Michal Kedzierski,et al.  A Method of Panchromatic Image Modification for Satellite Imagery Data Fusion , 2017, Remote. Sens..

[4]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[5]  Thomas Maurer.  HOW TO PAN-SHARPEN IMAGES USING THE GRAM-SCHMIDT PAN-SHARPEN METHOD – A RECIPE , 2013 .

[6]  Chengqi Cheng,et al.  A CNN-Based Pan-Sharpening Method for Integrating Panchromatic and Multispectral Images Using Landsat 8 , 2019, Remote. Sens..

[7]  Junfeng Yang,et al.  PanNet: A Deep Network Architecture for Pan-Sharpening , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[9]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Junho Yeom,et al.  Hybrid Pansharpening Algorithm for High Spatial Resolution Satellite Imagery to Improve Spatial Quality , 2013, IEEE Geoscience and Remote Sensing Letters.

[11]  Zhongping Sun,et al.  Joint Retrieval of Growing Season Corn Canopy LAI and Leaf Chlorophyll Content by Fusing Sentinel-2 and MODIS Images , 2019, Remote. Sens..

[12]  Hans Hagen,et al.  An Uncertainty-Aware Visual System for Image Pre-Processing , 2018, J. Imaging.

[13]  Dong Liang,et al.  Fusion of Unmanned Aerial Vehicle Panchromatic and Hyperspectral Images Combining Joint Skewness-Kurtosis Figures and a Non-Subsampled Contourlet Transform , 2018, Sensors.

[14]  Anna Fryskowska,et al.  SOME ASPECTS OF SATELLITE IMAGERY INTEGRATION FROM EROS B AND LANDSAT 8 , 2016 .

[15]  Jiemin Wu,et al.  Deep Self-Learning Network for Adaptive Pansharpening , 2019, Remote. Sens..

[16]  Aleksandra Sekrecka,et al.  Integration of Satellite Data with High Resolution Ratio: Improvement of Spectral Quality with Preserving Spatial Details , 2018, Sensors.

[17]  Ming Zhang,et al.  Multiresolution Bilateral Filtering for Image Denoising , 2008, IEEE Transactions on Image Processing.

[18]  Zhou Wang,et al.  Why is image quality assessment so difficult? , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Consuelo Gonzalo,et al.  A directed search algorithm for setting the spectral–spatial quality trade-off of fused images by the wavelet à trous method , 2008 .

[20]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[21]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Yunsong Li,et al.  Structure Tensor-Based Algorithm for Hyperspectral and Panchromatic Images Fusion , 2018, Remote. Sens..

[23]  Julien Radoux,et al.  Bayesian Data Fusion for Adaptable Image Pansharpening , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[24]  L. Wald,et al.  Fusion of high spatial and spectral resolution images : The ARSIS concept and its implementation , 2000 .

[25]  Firouz Abdullah Al-Wassai,et al.  The IHS Transformations Based Image Fusion , 2011, ArXiv.

[26]  Yonghyun Kim,et al.  Image Fusion of Spectrally Nonoverlapping Imagery Using SPCA and MTF-Based Filters , 2017, IEEE Geoscience and Remote Sensing Letters.

[27]  D. Holcomb,et al.  Optimizing the High-Pass Filter Addition Technique for Image Fusion , 2007 .

[28]  A. Orych,et al.  REVIEW OF METHODS FOR DETERMINING THE SPATIAL RESOLUTION OF UAV SENSORS , 2015 .

[29]  Laércio Massaru Namikawa,et al.  Image Fusion for Remote Sensing Applications , 2011 .

[30]  Luciano Alparone,et al.  Image fusion—the ARSIS concept and some successful implementation schemes , 2003 .

[31]  Javier Marcello,et al.  Evaluation of Spatial and Spectral Effectiveness of Pixel-Level Fusion Techniques , 2013, IEEE Geoscience and Remote Sensing Letters.

[32]  Timothy B. Terriberry,et al.  GPU Accelerating Speeded-Up Robust Features , 2008 .

[33]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[34]  D. Amarsaikhan,et al.  Fusing high-resolution SAR and optical imagery for improved urban land cover study and classification , 2010 .

[35]  Xiaohua Tong,et al.  A New Fusion Approach for Extracting Urban Built-up Areas from Multisource Remotely Sensed Data , 2019, Remote. Sens..

[36]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[37]  Hankui K. Zhang,et al.  A New Look at Image Fusion Methods from a Bayesian Perspective , 2015, Remote. Sens..

[38]  Stefano Baronti,et al.  On the Use of the Expanded Image in Quality Assessment of Pansharpened Images , 2018, IEEE Geoscience and Remote Sensing Letters.

[39]  W. Hargrove,et al.  Photogrammetric Engineering & Remote Sensing , 2022 .

[40]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[41]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[42]  Theo E. Schouten,et al.  Quality measures for image segmentation using generated images , 1995, Remote Sensing.

[43]  Agnieszka Jenerowicz,et al.  The fusion of satellite and UAV data: simulation of high spatial resolution band , 2017, Remote Sensing.

[44]  Gonzalo Pajares,et al.  A wavelet-based image fusion tutorial , 2004, Pattern Recognit..

[45]  Te-Ming Tu,et al.  Adjustable intensity-hue-saturation and Brovey transform fusion technique for IKONOS/ QuickBird imagery , 2005 .

[46]  B. K. Shreyamsha Kumar,et al.  Image denoising based on gaussian/bilateral filter and its method noise thresholding , 2013, Signal Image Video Process..

[47]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[48]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[49]  Yun Zhang,et al.  An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images , 2005, Inf. Fusion.

[50]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[51]  Consuelo Gonzalo Martín,et al.  A directed search algorithm for setting the spectral-spatial quality trade-off of fused images by the wavelet à trous method , 2008 .