Automatic Color Correction for Multisource Remote Sensing Images with Wasserstein CNN

In this paper a non-parametric model based on Wasserstein CNN is proposed for color correction. It is suitable for large-scale remote sensing image preprocessing from multiple sources under various viewing conditions, including illumination variances, atmosphere disturbances, and sensor and aspect angles. Color correction aims to alter the color palette of an input image to a standard reference which does not suffer from the mentioned disturbances. Most of current methods highly depend on the similarity between the inputs and the references, with respect to both the contents and the conditions, such as illumination and atmosphere condition. Segmentation is usually necessary to alleviate the color leakage effect on the edges. Different from the previous studies, the proposed method matches the color distribution of the input dataset with the references in a probabilistic optimal transportation framework. Multi-scale features are extracted from the intermediate layers of the lightweight CNN model and are utilized to infer the undisturbed distribution. The Wasserstein distance is utilized to calculate the cost function to measure the discrepancy between two color distributions. The advantage of the method is that no registration or segmentation processes are needed, benefiting from the local texture processing potential of the CNN models. Experimental results demonstrate that the proposed method is effective when the input and reference images are of different sources, resolutions, and under different illumination and atmosphere conditions.

[1]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[2]  A. Abadpour,et al.  A fast and efficient fuzzy color transfer method , 2004, Proceedings of the Fourth IEEE International Symposium on Signal Processing and Information Technology, 2004..

[3]  Hiroaki Kotera,et al.  A scene-referred color transfer for pleasant imaging on display , 2005, IEEE International Conference on Image Processing 2005.

[4]  A.C. Kokaram,et al.  N-dimensional probability density function transfer and its application to color transfer , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  Fabio Pellacini,et al.  User‐Controllable Color Transfer , 2010, Comput. Graph. Forum.

[6]  Ze-Nian Li,et al.  Semisupervised manifold learning for color transfer between multiview images , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[7]  Ján Morovic,et al.  Accurate 3D image colour histogram transformation , 2003, Pattern Recognit. Lett..

[8]  Robert B. Fisher,et al.  Recoding Color Transfer as A Color Homography , 2016, BMVC.

[9]  Jiann-Yeou Rau,et al.  True orthophoto generation of built-up areas using multi-view images , 2002 .

[10]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[12]  LiXuelong,et al.  Robust color correction in stereo vision , 2011 .

[13]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Erik Reinhard,et al.  Colour Spaces for Colour Transfer , 2011, CCIW.

[15]  C. Villani Optimal Transport: Old and New , 2008 .

[16]  Qi Wang,et al.  Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[17]  László Neumann,et al.  Color Style Transfer Techniques using Hue, Lightness and Saturation Histogram Matching , 2005, CAe.

[18]  Erik Reinhard,et al.  Progressive color transfer for images of arbitrary dynamic range , 2011, Comput. Graph..

[19]  Chi-Keung Tang,et al.  Local color transfer via probabilistic segmentation by expectation-maximization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[21]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[22]  David Avis,et al.  Ground metric learning , 2011, J. Mach. Learn. Res..

[23]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[24]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[25]  Anil Kokaram,et al.  The linear Monge-Kantorovitch linear colour mapping for example-based colour transfer , 2007 .

[26]  Kevin E. Bassler,et al.  Optimal transport on complex networks , 2007 .

[27]  Gregory Shakhnarovich,et al.  Learning Representations for Automatic Colorization , 2016, ECCV.

[28]  Yacov Hel-Or,et al.  Piecewise-consistent color mappings of images acquired under various conditions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Sylvain Paris,et al.  Example-based video color grading , 2013, ACM Trans. Graph..