Thick Cloud Removal in High-Resolution Satellite Images Using Stepwise Radiometric Adjustment and Residual Correction

Cloud cover is a common problem in optical satellite imagery, which leads to missing information in images as well as a reduction in the data usability. In this paper, a thick cloud removal method based on stepwise radiometric adjustment and residual correction (SRARC) is proposed, which is aimed at effectively removing the clouds in high-resolution images for the generation of high-quality and spatially contiguous urban geographical maps. The basic idea of SRARC is that the complementary information in adjacent temporal satellite images can be utilized for the seamless recovery of cloud-contaminated areas in the target image after precise radiometric adjustment. To this end, the SRARC method first optimizes the given cloud mask of the target image based on superpixel segmentation, which is conducted to ensure that the labeled cloud boundaries go through homogeneous areas of the target image, to ensure a seamless reconstruction. Stepwise radiometric adjustment is then used to adjust the radiometric information of the complementary areas in the auxiliary image, step by step, and clouds in the target image can be removed by the replacement with the adjusted complementary areas. Finally, residual correction based on global optimization is used to further reduce the radiometric differences between the recovered areas and the cloud-free areas. The final cloud removal results are then generated. High-resolution images with different spatial resolutions and land-cover change patterns were used in both simulated and real-data cloud removal experiments. The results suggest that SRARC can achieve a better performance than the other compared methods, due to the superiority of the radiometric adjustment and spatial detail preservation. SRARC is thus a promising approach that has the potential for routine use, to support applications based on high-resolution satellite images.

[1]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[2]  Gang Yang,et al.  On the Generation of Gapless and Seamless Daily Surface Reflectance Data , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Bin Chen,et al.  Spatially and Temporally Weighted Regression: A Novel Method to Produce Continuous Cloud-Free Landsat Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Gang Yang,et al.  Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Robert Eckardt,et al.  Removal of Optically Thick Clouds from Multi-Spectral Satellite Images Using Multi-Frequency SAR Data , 2013, Remote. Sens..

[7]  Chao Zeng,et al.  Missing Data Reconstruction in Remote Sensing Image With a Unified Spatial–Temporal–Spectral Deep Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[9]  Yanfeng Gu,et al.  Multitemporal Landsat Missing Data Recovery Based on Tempo-Spectral Angle Model , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[10]  András Bárdossy,et al.  Cloud removal methodology from MODIS snow cover product , 2009 .

[11]  Liangpei Zhang,et al.  Inpainting for Remotely Sensed Images With a Multichannel Nonlocal Total Variation Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Chenghu Zhou,et al.  A Sparse Dictionary Learning-Based Adaptive Patch Inpainting Method for Thick Clouds Removal from High-Spatial Resolution Remote Sensing Imagery , 2017, Sensors.

[13]  Yang Hong,et al.  A two-step framework for reconstructing remotely sensed land surface temperatures contaminated by cloud , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[14]  Ryutaro Tateishi,et al.  Cloud Removal of Optical Image Using SAR Data for ALOS Applications. Experimenting on Simulated ALOS Data , 2009 .

[15]  David Morin,et al.  Operational High Resolution Land Cover Map Production at the Country Scale Using Satellite Image Time Series , 2017, Remote. Sens..

[16]  Liangpei Zhang,et al.  Missing Information Reconstruction for Single Remote Sensing Images Using Structure-Preserving Global Optimization , 2017, IEEE Signal Processing Letters.

[17]  Jin Chen,et al.  A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky-Golay filter , 2004 .

[18]  Chao Zeng,et al.  Recovering missing pixels for Landsat ETM + SLC-off imagery using multi-temporal regression analysis and a regularization method , 2013 .

[19]  José A. Sobrino,et al.  Comparison of cloud-reconstruction methods for time series of composite NDVI data , 2010 .

[20]  Zhihao Qin,et al.  An Efficient Approach to Remove Thick Cloud in VNIR Bands of Multi-Temporal Remote Sensing Images , 2019, Remote. Sens..

[21]  Matthew F. McCabe,et al.  Reconstructing Cloud Contaminated Pixels Using Spatiotemporal Covariance Functions and Multitemporal Hyperspectral Imagery , 2019, Remote. Sens..

[22]  Zhiwei Li,et al.  Deep learning based cloud detection for remote sensing images by the fusion of multi-scale convolutional features , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[23]  Xinhao Liu,et al.  Single-Image Noise Level Estimation for Blind Denoising , 2013, IEEE Transactions on Image Processing.

[24]  Farid Melgani,et al.  Missing-Area Reconstruction in Multispectral Images Under a Compressive Sensing Perspective , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Ying Li,et al.  Cloud Removal From Optical Satellite Imagery With SAR Imagery Using Sparse Representation , 2015, IEEE Geoscience and Remote Sensing Letters.

[26]  Weidong Li,et al.  Gaps‐fill of SLC‐off Landsat ETM+ satellite image using a geostatistical approach , 2007 .

[27]  Liangpei Zhang,et al.  Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model , 2014 .

[28]  Xinghua Li,et al.  Cloud removal in remote sensing images using nonnegative matrix factorization and error correction , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[29]  Feng Gao,et al.  A Modified Neighborhood Similar Pixel Interpolator Approach for Removing Thick Clouds in Landsat Images , 2012, IEEE Geoscience and Remote Sensing Letters.

[30]  Gang Yang,et al.  Missing Information Reconstruction of Remote Sensing Data: A Technical Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[31]  Keshav Prasad Paudel,et al.  Monitoring snow cover variability in an agropastoral area in the Trans Himalayan region of Nepal using MODIS data with improved cloud removal methodology , 2011 .

[32]  Farid Melgani,et al.  Contextual reconstruction of cloud-contaminated multitemporal multispectral images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Collin G. Homer,et al.  Automated cloud and shadow detection and filling using two-date Landsat imagery in the USA , 2013 .

[34]  Arvind Singh,et al.  Optical Cloud Pixel Recovery via Machine Learning , 2017, Remote. Sens..

[35]  Antonio J. Plaza,et al.  Cloud Removal Based on Sparse Representation via Multitemporal Dictionary Learning , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Brian Johnson,et al.  Classifying a high resolution image of an urban area using super-object information , 2013 .

[37]  Ying Li,et al.  Removal of Optically Thick Clouds From High-Resolution Satellite Imagery Using Dictionary Group Learning and Interdictionary Nonlocal Joint Sparse Coding , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[38]  Weidong Li,et al.  Restoration of clouded pixels in multispectral remotely sensed imagery with cokriging , 2009 .

[39]  Zhiwei Li,et al.  A Spatiotemporal Fusion Based Cloud Removal Method for Remote Sensing Images With Land Cover Changes , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[40]  J. Storey,et al.  LANDSAT 7 SCAN LINE CORRECTOR-OFF GAP-FILLED PRODUCT DEVELOPMENT , 2005 .

[41]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[42]  Francesca Bovolo,et al.  Automatic cloud-free image generation from high-resolution multitemporal imagery , 2017 .

[43]  Per Jönsson,et al.  Seasonality extraction by function fitting to time-series of satellite sensor data , 2002, IEEE Trans. Geosci. Remote. Sens..

[44]  Chao-Hung Lin,et al.  Patch-Based Information Reconstruction of Cloud-Contaminated Multitemporal Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Christine Fernandez-Maloigne,et al.  A Bandelet-Based Inpainting Technique for Clouds Removal From Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.