Missing Information Reconstruction of Remote Sensing Data: A Technical Review

Because of sensor malfunction and poor atmospheric conditions, there is usually a great deal of missing information in optical remote sensing data, which reduces the usage rate and hinders the follow-up interpretation. In the past decades, missing information reconstruction of remote sensing data has become an active research field, and a large number of algorithms have been developed. However, to the best of our knowledge, there has not, to date, been a study that has been aimed at expatiating and summarizing the current situation. This is therefore our motivation in this review. This paper provides an introduction to the principles and theories of missing information reconstruction of remote sensing data. We classify the established and emerging algorithms into four main categories, followed by a comprehensive comparison of them from both experimental and theoretical perspectives. This paper also predicts the promising future research directions.

[1]  W. Verhoef,et al.  Reconstructing cloudfree NDVI composites using Fourier analysis of time series , 2000 .

[2]  Célia A. Zorzo Barcelos,et al.  Image inpainting and denoising by nonlinear partial differential equations , 2003, 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2003).

[3]  Freek D. van der Meer,et al.  Remote-sensing image analysis and geostatistics , 2012 .

[4]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[5]  Chao Yu,et al.  Kriging interpolation method and its application in retrieval of MODIS aerosol optical depth , 2011, 2011 19th International Conference on Geoinformatics.

[6]  Liangpei Zhang,et al.  A Principal Component Based Haze Masking Method for Visible Images , 2014, IEEE Geoscience and Remote Sensing Letters.

[7]  Xiaoli Ding,et al.  Reconstruction of Time-Series MODIS LST in Central Qinghai-Tibet Plateau Using Geostatistical Approach , 2013, IEEE Geoscience and Remote Sensing Letters.

[8]  Jun Zhang,et al.  Missing Data and Regression Models for Spatial Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[9]  William J. Emery,et al.  Active Learning Methods for Remote Sensing Image Classification , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Liangpei Zhang,et al.  Sparse-based reconstruction of missing information in remote sensing images from spectral/temporal complementary information , 2015 .

[11]  Feng Gao,et al.  A simple and effective method for filling gaps in Landsat ETM+ SLC-off images , 2011 .

[12]  John L. Dwyer,et al.  Development of the Landsat Data Continuity Mission Cloud-Cover Assessment Algorithms , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Nikos Komodakis,et al.  Image Completion Using Global Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  Wataru Takeuchi,et al.  Restoration of Aqua MODIS Band 6 Using Histogram Matching and Local Least Squares Fitting , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Weidong Li,et al.  Gaps‐fill of SLC‐off Landsat ETM+ satellite image using a geostatistical approach , 2007 .

[16]  Roi Méndez-Rial,et al.  Anisotropic Inpainting of the Hypercube , 2012, IEEE Geoscience and Remote Sensing Letters.

[17]  Liangpei Zhang,et al.  Hyperspectral Image Denoising Employing a Spectral–Spatial Adaptive Total Variation Model , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[18]  S. Goward,et al.  Characterization of the Landsat-7 ETM Automated Cloud-Cover Assessment (ACCA) Algorithm , 2006 .

[19]  David B. Dunson,et al.  Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images , 2012, IEEE Transactions on Image Processing.

[20]  Jun Zhang,et al.  Functional Concurrent Linear Regression Model for Spatial Images , 2011 .

[21]  Manuel Menezes de Oliveira Neto,et al.  Fast Digital Image Inpainting , 2001, VIIP.

[22]  Hongjie Xie,et al.  Integrated assessment on multi-temporal and multi-sensor combinations for reducing cloud obscuration of MODIS snow cover products of the Pacific Northwest USA , 2010 .

[23]  Per Jönsson,et al.  TIMESAT - a program for analyzing time-series of satellite sensor data , 2004, Comput. Geosci..

[24]  Jian Sun,et al.  Statistics of Patch Offsets for Image Completion , 2012, ECCV.

[25]  Liangpei Zhang,et al.  Compressed Sensing-Based Inpainting of Aqua Moderate Resolution Imaging Spectroradiometer Band 6 Using Adaptive Spectrum-Weighted Sparse Bayesian Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Bo Huang,et al.  Unified fusion of remote-sensing imagery: generating simultaneously high-resolution synthetic spatial–temporal–spectral earth observations , 2013 .

[27]  Christophe Latry,et al.  Cloud detection with SVM technique , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[28]  José A. Sobrino,et al.  Global land surface phenology trends from GIMMS database , 2009 .

[29]  Chao Zeng,et al.  Restoring Aqua MODIS band 6 by other spectral bands using compressed sensing theory , 2012, 2012 4th Workshop on Hyperspectral Image and Signal Processing (WHISPERS).

[30]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[31]  Jordi Inglada,et al.  Land-cover maps from partially cloudy multi-temporal image series: Optimal temporal sampling and cloud removal , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[32]  Liangpei Zhang,et al.  Recovering Reflectance of AQUA MODIS Band 6 Based on Within-Class Local Fitting , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[33]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[34]  Carl-Fredrik Westin,et al.  A Novel Gauss-Markov Random Field Approach for Regularization of Diffusion Tensor Maps , 2003, EUROCAST.

[35]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[37]  Bo Huang,et al.  Comparison of three time-series NDVI reconstruction methods based on TIMESAT , 2012, IGARSS.

[38]  D. Roy,et al.  Multi-temporal MODIS-Landsat data fusion for relative radiometric normalization, gap filling, and prediction of Landsat data , 2008 .

[39]  Zhe Zhu,et al.  Object-based cloud and cloud shadow detection in Landsat imagery , 2012 .

[40]  Huifang Li,et al.  An effective thin cloud removal procedure for visible remote sensing images , 2014 .

[41]  Chuanmin Hu,et al.  A Hybrid Cloud Detection Algorithm to Improve MODIS Sea Surface Temperature Data Quality and Coverage Over the Eastern Gulf of Mexico , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[42]  P. Beck,et al.  Improved monitoring of vegetation dynamics at very high latitudes: A new method using MODIS NDVI , 2006 .

[43]  Chun-Liang Chien,et al.  Automatic cloud removal from multi-temporal SPOT images , 2008, Appl. Math. Comput..

[44]  Liangpei Zhang,et al.  Single image haze removal considering sensor blur and noise , 2013, EURASIP J. Adv. Signal Process..

[45]  Hao He,et al.  A Changing-Weight Filter Method for Reconstructing a High-Quality NDVI Time Series to Preserve the Integrity of Vegetation Phenology , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[46]  F. Veroustraete,et al.  Reconstructing pathfinder AVHRR land NDVI time-series data for the Northwest of China , 2006 .

[47]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[48]  Huifang Li,et al.  A Variational Gradient-based Fusion Method for Visible and SWIR Imagery , 2012 .

[49]  Liangpei Zhang,et al.  Inpainting for Remotely Sensed Images With a Multichannel Nonlocal Total Variation Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Liangpei Zhang,et al.  Analysis model based recovery of remote sensing data , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[51]  Jin Chen,et al.  A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky-Golay filter , 2004 .

[52]  B. Holben.  Characteristics of maximum-value composite images from temporal AVHRR data , 1986 .

[53]  Liangpei Zhang,et al.  A MAP-Based Algorithm for Destriping and Inpainting of Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Yi Wang,et al.  Universal reconstruction method for radiometric quality improvement of remote sensing images , 2010, Int. J. Appl. Earth Obs. Geoinformation.

[55]  A. Belward,et al.  The Best Index Slope Extraction ( BISE): A method for reducing noise in NDVI time-series , 1992 .

[56]  Farid Melgani,et al.  Assessing the Reconstructability of Shadow Areas in VHR Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Nianzeng Che,et al.  A new method for retrieving band 6 of aqua MODIS , 2006, IEEE Geoscience and Remote Sensing Letters.

[58]  Zhe Zhu,et al.  Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data: An algorithm designed specifically for monitoring land cover change , 2014 .

[59]  José A. Sobrino,et al.  Comparison of cloud-reconstruction methods for time series of composite NDVI data , 2010 .

[60]  Chao-Hung Lin,et al.  Cloud Removal From Multitemporal Satellite Images Using Information Cloning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Liangpei Zhang,et al.  Reconstructing MODIS LST Based on Multitemporal Classification and Robust Regression , 2015, IEEE Geoscience and Remote Sensing Letters.

[62]  Tim R. McVicar,et al.  Climate‐related trends in Australian vegetation cover as inferred from satellite observations, 1981–2006 , 2009 .

[63]  Farid Melgani,et al.  Contextual reconstruction of cloud-contaminated multitemporal multispectral images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[64]  Michael Schmidt,et al.  Geostatistical interpolation of SLC-off Landsat ETM+ images , 2009 .

[65]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[66]  R. Lark,et al.  Geostatistics for Environmental Scientists , 2001 .

[67]  J. Storey,et al.  LANDSAT 7 SCAN LINE CORRECTOR-OFF GAP-FILLED PRODUCT DEVELOPMENT , 2005 .

[68]  Liangpei Zhang,et al.  Dead Pixel Completion of Aqua MODIS Band 6 Using a Robust M-Estimator Multiregression , 2014, IEEE Geoscience and Remote Sensing Letters.

[69]  Xiaoliang Lu,et al.  Removal of Noise by Wavelet Method to Generate High Quality Temporal Data of Terrestrial MODIS Products , 2007 .

[70]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[71]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[72]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..

[73]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[74]  Michael D. Grossberg,et al.  Quantitative Restoration for MODIS Band 6 on Aqua , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[75]  Chao Zeng,et al.  Recovering missing pixels for Landsat ETM + SLC-off imagery using multi-temporal regression analysis and a regularization method , 2013 .

[76]  Chao-Hung Lin,et al.  Patch-Based Information Reconstruction of Cloud-Contaminated Multitemporal Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[77]  Christine Fernandez-Maloigne,et al.  A Bandelet-Based Inpainting Technique for Clouds Removal From Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[78]  C. Justice,et al.  The generation of global fields of terrestrial biophysical parameters from the NDVI , 1994 .

[79]  Soo Chin Liew,et al.  Producing cloud free and cloud-shadow free mosaic from cloudy IKONOS images , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[80]  Christine Guillemot,et al.  Image Inpainting : Overview and Recent Advances , 2014, IEEE Signal Processing Magazine.

[81]  Liangpei Zhang,et al.  Adjustable Model-Based Fusion Method for Multispectral and Panchromatic Images , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[82]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[83]  W. Verhoef,et al.  A colour composite of NOAA-AVHRR-NDVI based on time series analysis (1981-1992) , 1996 .

[84]  Grégoire Mercier,et al.  Preprocessing of Low-Resolution Time Series Contaminated by Clouds and Shadows , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[85]  Gang Yang,et al.  Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[86]  Liangpei Zhang,et al.  An Adaptive Nonlocal Regularized Shadow Removal Method for Aerial Remote Sensing Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[87]  Liangpei Zhang,et al.  Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model , 2014 .

[88]  J. Townshend,et al.  Spatially and temporally continuous LAI data sets based on an integrated filtering method: Examples from North America , 2008 .

[89]  E. Helmer,et al.  Cloud-Free Satellite Image Mosaics with Regression Trees and Histogram Matching. , 2005 .

[90]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[91]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[92]  Farid Melgani,et al.  Missing-Area Reconstruction in Multispectral Images Under a Compressive Sensing Perspective , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[93]  Y. Kerr,et al.  On current compositing algorithms , 1997 .

[94]  Per Jönsson,et al.  Seasonality extraction by function fitting to time-series of satellite sensor data , 2002, IEEE Trans. Geosci. Remote. Sens..

[95]  Farid Melgani,et al.  Contextual Spatiospectral Postreconstruction of Cloud-Contaminated Images , 2008, IEEE Geoscience and Remote Sensing Letters.

[96]  Guillermo Sapiro,et al.  A Comprehensive Framework for Image Inpainting , 2010, IEEE Transactions on Image Processing.

[97]  Frédéric Baret,et al.  The CACAO Method for Smoothing, Gap Filling, and Characterizing Seasonal Anomalies in Satellite Time Series , 2013, IEEE Transactions on Geoscience and Remote Sensing.