One-to-one Example-based Automatic Image Coloring Using Deep Convolutional Generative Adversarial Network

Due to the indeterminate nature of the problem, image colorization techniques currently rely heavily on human intuition. Using deep convolutional networks, we can build a system that takes a source image to guide localdependent feature color mapping and color a grayscale target image. Unlike most other convolutional neural network approaches that require a lot of training data, our proposed system uses only one image for training for each target image. Our system is based on deep convolutional generative adversarial networks, which contains concepts of both supervised and unsupervised learning. We proposed a model architecture, objective functions, and both preprocessing and postprocessing algorithms for the image coloring process. We evaluated our system on a variety of input images and showed that it produce excellent results. 

[1]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[2]  Stefano Soatto,et al.  Quick Shift and Kernel Methods for Mode Seeking , 2008, ECCV.

[3]  Xiang Zhang,et al.  Automatic grayscale image colorization using histogram regression , 2012, Pattern Recognit. Lett..

[4]  Bernhard Schölkopf,et al.  Automatic Image Colorization Via Multimodal Predictions , 2008, ECCV.

[5]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[6]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[7]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[11]  Hiroshi Ishikawa,et al.  Let there be color! , 2016, ACM Trans. Graph..

[12]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Deepu Rajan,et al.  Image colorization using similar images , 2012, ACM Multimedia.

[15]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[16]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[17]  Dongdong Nie,et al.  Optimization based grayscale image colorization , 2007, Pattern Recognit. Lett..

[18]  Guillermo Sapiro,et al.  Fast image and video colorization using chrominance blending , 2006, IEEE Transactions on Image Processing.