Nonlocal Tensor Completion for Multitemporal Remotely Sensed Images’ Inpainting

Remotely sensed images may contain some missing areas because of poor weather conditions and sensor failure. Information of those areas may play an important role in the interpretation of multitemporal remotely sensed data. This paper aims at reconstructing the missing information by a nonlocal low-rank tensor completion method. First, nonlocal correlations in the spatial domain are taken into account by searching and grouping similar image patches in a large search window. Then, low rankness of the identified fourth-order tensor groups is promoted to consider their correlations in spatial, spectral, and temporal domains, while reconstructing the underlying patterns. Experimental results on simulated and real data demonstrate that the proposed method is effective both qualitatively and quantitatively. In addition, the proposed method is computationally efficient compared with other patch-based methods such as the recently proposed patch matching-based multitemporal group sparse representation method.

[1]  Wataru Takeuchi,et al.  Restoration of Aqua MODIS Band 6 Using Histogram Matching and Local Least Squares Fitting , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Chao Yu,et al.  Kriging interpolation method and its application in retrieval of MODIS aerosol optical depth , 2011, 2011 19th International Conference on Geoinformatics.

[3]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[4]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Liangpei Zhang,et al.  Inpainting for Remotely Sensed Images With a Multichannel Nonlocal Total Variation Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[7]  Gang Yang,et al.  Missing Information Reconstruction of Remote Sensing Data: A Technical Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[8]  Gary A. Shaw,et al.  Hyperspectral Image Processing for Automatic Target Detection Applications , 2003 .

[9]  Xiao Xiang Zhu,et al.  Robust Object-Based Multipass InSAR Deformation Reconstruction , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Zhenhua Li,et al.  Color transfer based remote sensing image fusion using non-separable wavelet frame transform , 2005, Pattern Recognit. Lett..

[11]  Xiao Xiang Zhu,et al.  Object-Based Multipass InSAR via Robust Low-Rank Tensor Decomposition , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Michael K. Ng,et al.  A New Convex Optimization Model for Multiplicative Noise and Blur Removal , 2014, SIAM J. Imaging Sci..

[14]  Gang Yang,et al.  Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Richard G. Baraniuk,et al.  Sparsity and Structure in Hyperspectral Imaging : Sensing, Reconstruction, and Target Detection , 2014, IEEE Signal Processing Magazine.

[17]  Naoto Yokoya,et al.  Hyperspectral Tree Species Classification of Japanese Complex Mixed Forest With the Aid of Lidar Data , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Liangpei Zhang,et al.  Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model , 2014 .

[19]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[20]  Melba M. Crawford,et al.  Manifold-Learning-Based Feature Extraction for Classification of Hyperspectral Data: A Review of Advances in Manifold Learning , 2014, IEEE Signal Processing Magazine.

[21]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[22]  Liangpei Zhang,et al.  Recovering Reflectance of AQUA MODIS Band 6 Based on Within-Class Local Fitting , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[23]  Peder A. Olsen,et al.  Removing Clouds and Recovering Ground Observations in Satellite Image Sequences via Temporally Contiguous Robust Matrix Completion , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  José A. Sobrino,et al.  Comparison of cloud-reconstruction methods for time series of composite NDVI data , 2010 .

[25]  Michael K. Ng,et al.  Deblurring and Sparse Unmixing for Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Farid Melgani,et al.  Contextual Spatiospectral Postreconstruction of Cloud-Contaminated Images , 2008, IEEE Geoscience and Remote Sensing Letters.

[27]  Xiao Xiang Zhu,et al.  Unsupervised Spectral–Spatial Feature Learning via Deep Residual Conv–Deconv Network for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Guillermo Sapiro,et al.  A Comprehensive Framework for Image Inpainting , 2010, IEEE Transactions on Image Processing.

[29]  Liangpei Zhang,et al.  Reconstructing MODIS LST Based on Multitemporal Classification and Robust Regression , 2015, IEEE Geoscience and Remote Sensing Letters.

[30]  Liangpei Zhang,et al.  Patch Matching-Based Multitemporal Group Sparse Representation for the Missing Information Reconstruction of Remote-Sensing Images , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  S. Gabarda,et al.  Blind image quality assessment through anisotropy. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[32]  Hao He,et al.  A Changing-Weight Filter Method for Reconstructing a High-Quality NDVI Time Series to Preserve the Integrity of Vegetation Phenology , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Jon Atli Benediktsson,et al.  SVM- and MRF-Based Method for Accurate Classification of Hyperspectral Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[34]  Xiaoxiang Zhu,et al.  Automatic Detection and Reconstruction of 2-D/3-D Building Shapes From Spaceborne TomoSAR Point Clouds , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[36]  Liangpei Zhang,et al.  Compressed Sensing-Based Inpainting of Aqua Moderate Resolution Imaging Spectroradiometer Band 6 Using Adaptive Spectrum-Weighted Sparse Bayesian Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Naoto Yokoya,et al.  Detection and correction of spectral and spatial misregistrations for hyperspectral data using phase correlation method. , 2010, Applied optics.

[38]  Farid Melgani,et al.  Missing-Area Reconstruction in Multispectral Images Under a Compressive Sensing Perspective , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Antonio J. Plaza,et al.  Sparse Unmixing of Hyperspectral Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Jon Atli Benediktsson,et al.  Extinction Profiles for the Classification of Remote Sensing Data , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Chao Zeng,et al.  Recovering missing pixels for Landsat ETM + SLC-off imagery using multi-temporal regression analysis and a regularization method , 2013 .

[42]  Liangpei Zhang,et al.  Total-Variation-Regularized Low-Rank Matrix Factorization for Hyperspectral Image Restoration , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Chao-Hung Lin,et al.  Patch-Based Information Reconstruction of Cloud-Contaminated Multitemporal Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Christine Fernandez-Maloigne,et al.  A Bandelet-Based Inpainting Technique for Clouds Removal From Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[45]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[46]  Liangpei Zhang,et al.  A MAP-Based Algorithm for Destriping and Inpainting of Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Liangpei Zhang,et al.  Sparse-based reconstruction of missing information in remote sensing images from spectral/temporal complementary information , 2015 .

[48]  Yi Wang,et al.  Universal reconstruction method for radiometric quality improvement of remote sensing images , 2010, Int. J. Appl. Earth Obs. Geoinformation.

[49]  Nasser M. Nasrabadi,et al.  Hyperspectral Target Detection : An Overview of Current and Future Challenges , 2014, IEEE Signal Processing Magazine.

[50]  Michael D. Grossberg,et al.  Quantitative Restoration for MODIS Band 6 on Aqua , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Naoto Yokoya,et al.  Object Detection Based on Sparse Representation and Hough Voting for Optical Remote Sensing Imagery , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Naoto Yokoya,et al.  Nonlinear Unmixing of Hyperspectral Data Using Semi-Nonnegative Matrix Factorization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Olivier Hagolle,et al.  MACCS-ATCOR joint algorithm (MAJA) , 2016, Remote Sensing.

[54]  Peter Reinartz,et al.  Joint Sparsity Model for Multilook Hyperspectral Image Unmixing , 2015, IEEE Geoscience and Remote Sensing Letters.

[55]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[56]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[57]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[58]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[59]  Nianzeng Che,et al.  A new method for retrieving band 6 of aqua MODIS , 2006, IEEE Geoscience and Remote Sensing Letters.

[60]  Chein-I Chang,et al.  Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach , 1994, IEEE Trans. Geosci. Remote. Sens..

[61]  Gang Liu,et al.  Tensor completion using total variation and low-rank matrix factorization , 2016, Inf. Sci..

[62]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Tingzhu Huang,et al.  A non-convex tensor rank approximation for tensor completion , 2017 .

[64]  Xiang Zhu,et al.  Automatic Parameter Selection for Denoising Algorithms Using a No-Reference Measure of Image Content , 2010, IEEE Transactions on Image Processing.

[65]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Liangpei Zhang,et al.  Dead Pixel Completion of Aqua MODIS Band 6 Using a Robust M-Estimator Multiregression , 2014, IEEE Geoscience and Remote Sensing Letters.

[67]  Guangming Shi,et al.  Compressive Sensing via Nonlocal Low-Rank Regularization , 2014, IEEE Transactions on Image Processing.