Artistic photo filter removal using convolutional neural networks

Abstract. We present a method for the automatic restoration of images subjected to the application of photographic filters, such as those made popular by photo-sharing services. The method uses a convolutional neural network (CNN) for the prediction of the coefficients of local polynomial transformations that are applied to the input image. The experiments we conducted on a subset of the Places-205 dataset show that the quality of the restoration performed by our method is clearly superior to that of traditional color balancing and restoration procedures, and to that of recent CNN architectures for image-to-image translation.

[1]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[2]  Claudio Cusano,et al.  Single and Multiple Illuminant Estimation Using Convolutional Neural Networks , 2015, IEEE Transactions on Image Processing.

[3]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[4]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[5]  Hiroshi Ishikawa,et al.  Let there be color! , 2016, ACM Trans. Graph..

[6]  Henry R. Kang.  Color Technology for Electronic Imaging Devices , 1997 .

[7]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Brian A. Wandell,et al.  A spatial extension of CIELAB for digital color‐image reproduction , 1997 .

[12]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Leonardo Vanneschi,et al.  Polynomial modeling and optimization for colorimetric characterization of scanners , 2008, J. Electronic Imaging.

[14]  Raimondo Schettini,et al.  Artistic Photo Filtering Recognition Using CNNs , 2017, CCIW.

[15]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[16]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[17]  Lou,et al.  UvA-DARE (Digital Academic Repository) Color Constancy by Deep Learning Color Constancy by Deep Learning , 2015 .

[18]  Seoung Wug Oh,et al.  Approaching the computational color constancy as a classification problem through deep learning , 2016, Pattern Recognit..

[19]  Raimondo Schettini,et al.  Error-Tolerant Color Rendering for Digital Cameras , 2014, Journal of Mathematical Imaging and Vision.

[20]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[21]  Gregory Shakhnarovich,et al.  Learning Representations for Automatic Colorization , 2016, ECCV.

[22]  Kristen Grauman,et al.  On-demand Learning for Deep Image Restoration , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Luc Van Gool,et al.  DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Xiaoou Tang,et al.  Deep Specialized Network for Illuminant Estimation , 2016, ECCV.

[25]  Wen-Chin Chen,et al.  Filter-Invariant Image Classification on Social Media Photos , 2015, ACM Multimedia.

[26]  Ming-Hsuan Yang,et al.  Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network , 2016, ECCV.

[27]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[28]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[29]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).