CGANs Based User Preferred Photorealistic Re-stylization of Social Image

In social networks, it is important to re-stylize a randomly taken photo to exhibit a unique individual character. Previous stylization methods either respect to a motivation of improving perceptual quality or artistic style transfer, are neither personalized nor photorealistic. Besides, a strong constraint on scene consistency of reference image is always required, which is not easy to meet for a customized application. In this paper, we propose a customized photorealistic re-stylization method referred to a group of user favorite images with loose scene consistency. To better express user preferred style, reference images are selected from the perspective of photographer where image content and composition are jointly considered and weighed by user preference of light and color. To achieve high perceptual quality, we map image pixels and styles based on Conditional Generative Adversarial Networks. Comprehensive experiments verify our method could improve user preferred photo re-stylization and bring in less artificiality.

[1]  Jong-Sen Lee,et al.  Digital Image Enhancement and Noise Filtering by Use of Local Statistics , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[3]  Neil A. Thacker,et al.  The Bhattacharyya metric as an absolute similarity measure for frequency coded data , 1998, Kybernetika.

[4]  Sylvain Paris,et al.  Example-based video color grading , 2013, ACM Trans. Graph..

[5]  Dani Lischinski,et al.  Personalization of image enhancement , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[7]  Hristina Hristova,et al.  Style-aware robust color transfer , 2015, CAE '15.

[8]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[10]  Bin Sheng,et al.  Deep Colorization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..

[12]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[13]  Mu Qiao,et al.  OSCAR: On-Site Composition and Aesthetics Feedback Through Exemplars for Photographers , 2012, International Journal of Computer Vision.

[14]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[15]  Dani Lischinski,et al.  Collaborative Personalization of Image Enhancement , 2011, CVPR 2011.

[16]  Chang Wen Chen,et al.  Color Photo Makeover via Crowd Sourcing and Recoloring , 2015, ACM Multimedia.

[17]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Erik Reinhard,et al.  A Survey of Color Mapping and its Applications , 2014, Eurographics.