Restoration of Sea Surface Temperature Satellite Images Using a Partially Occluded Training Set

Sea surface temperature(SST) satellite images are often partially occluded by clouds. Image inpainting is one approach to restore the occluded region. Considering the sparseness of SST images, they can be restored via learning-based inpainting. However, state-of-the-art learning-based inpainting methods using deep neural networks require large amount of non-occluded images as a training set. Since most SST images contain occluded regions, it is hard to collect sufficient non-occluded images. In this paper, we propose a novel method that uses occluded images as training images hence we can enlarge the amount of available training images from a certain SST image set. This is realized by comprising a novel reconstruction loss and adversarial loss. Experimental results confirm the effectiveness of our method.

[1]  Michael Elad,et al.  Learning Multiscale Sparse Representations for Image and Video Restoration , 2007, Multiscale Model. Simul..

[2]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[3]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2008, Commun. ACM.

[4]  Honglak Lee,et al.  Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising , 2013, NIPS.

[5]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[6]  Smith,et al.  Satellite measurements of sea surface temperature through clouds , 2000, Science.

[7]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Haruma Ishida,et al.  Development of an unbiased cloud detection algorithm for a spaceborne multispectral imager , 2009 .

[9]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  A. Mcnally,et al.  A cloud detection algorithm for high‐spectral‐resolution infrared sounders , 2003 .

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[14]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[15]  Misako Kachi,et al.  Sea surface temperature from the new Japanese geostationary meteorological Himawari‐8 satellite , 2016 .

[16]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Yosuke Fujii,et al.  Meteorological research institute multivariate ocean variational estimation (MOVE) system : Some early results , 2006 .

[18]  Robert H. Evans,et al.  A principal component analysis of sea-surface temperature in the Arabian Sea , 2001 .

[19]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[20]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[21]  Aline Roumy,et al.  Exemplar-based image inpainting: Fast priority and coherent nearest neighbor search , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[22]  Prashant D. Sardeshmukh,et al.  The Optimal Growth of Tropical Sea Surface Temperature Anomalies , 1995 .

[23]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[24]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Thomas M. Smith,et al.  Daily High-Resolution-Blended Analyses for Sea Surface Temperature , 2007 .