Unsupervised Diverse Colorization via Generative Adversarial Networks

Colorization of grayscale images is a hot topic in computer vision. Previous research mainly focuses on producing a color image to recover the original one in a supervised learning fashion. However, since many colors share the same gray value, an input grayscale image could be diversely colorized while maintaining its reality. In this paper, we design a novel solution for unsupervised diverse colorization. Specifically, we leverage conditional generative adversarial networks to model the distribution of real-world item colors, in which we develop a fully convolutional generator with multi-layer noise to enhance diversity, with multi-layer condition concatenation to maintain reality, and with stride 1 to keep spatial information. With such a novel network architecture, the model yields highly competitive performance on the open LSUN bedroom dataset. The Turing test on 80 humans further indicates our generated color schemes are highly convincible.

[1]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[2]  S. Laycock,et al.  Siggraph 2002 , 2002 .

[3]  SIGGRAPH 2001 , 2002 .

[4]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[5]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[6]  Tien-Tsin Wong,et al.  Manga colorization , 2006, ACM Trans. Graph..

[7]  Stephen Lin,et al.  Intrinsic colorization , 2008, ACM Trans. Graph..

[8]  Bernhard Schölkopf,et al.  Automatic Image Colorization Via Multimodal Predictions , 2008, ECCV.

[9]  Stephen Lin,et al.  Semantic colorization with internet images , 2011, ACM Trans. Graph..

[10]  Deepu Rajan,et al.  Image colorization using similar images , 2012, ACM Multimedia.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Bin Sheng,et al.  Deep Colorization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  David A. Forsyth,et al.  Learning Large-Scale Automatic Image Colorization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Hendrik P. A. Lensch,et al.  Infrared Colorization Using Deep Convolutional Neural Networks , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[17]  Bogdan Raducanu,et al.  Invertible Conditional GANs for image editing , 2016, ArXiv.

[18]  Myungjoo Kang,et al.  Variational Image Colorization Models Using Higher-Order Mumford–Shah Regularizers , 2016, J. Sci. Comput..

[19]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[20]  Ruck Thawonmas,et al.  Image Colorization Using a Deep Convolutional Neural Network , 2016, ArXiv.

[21]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[22]  Stephen Koo,et al.  Automatic Colorization with Deep Convolutional Generative Adversarial Networks , 2016 .

[23]  Hiroshi Ishikawa,et al.  Let there be color! , 2016, ACM Trans. Graph..

[24]  Terumasa Aoki,et al.  Automatic Image Colorization based on Feature Lines , 2016, VISIGRAPP.

[25]  Aditya Deshpande,et al.  Learning Diverse Image Colorization , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[27]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yike Guo,et al.  Unsupervised Image-to-Image Translation with Generative Adversarial Networks , 2017, ArXiv.

[29]  Frank Gabel.  Generative Adversarial Text-to-Image Synthesis , 2018 .