Image Recoloring with Valence‐Arousal Emotion Model

We introduce an affective image recoloring method for changing the overall mood in the image in a numerically measurable way. Given a semantically segmented source image and a target emotion, our system finds reference image segments from the collection of images that have been tagged via crowdsourcing with numerically measured emotion labels. We then recolorize the source segments using colors from the selected target segments while preserving the gradient of the source image to generate a seamless and natural result. User study confirms the effectiveness of our method in accomplishing the stated goal of altering the mood of the image to match the target emotion level.

[1]  Björn W. Schuller,et al.  Categorical and dimensional affect analysis in continuous input: Current trends and future directions , 2013, Image Vis. Comput..

[2]  Pat Hanrahan,et al.  Probabilistic color-by-numbers , 2013, ACM Trans. Graph..

[3]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[4]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[6]  Heesoo Myeong,et al.  Tensor-Based High-Order Semantic Relation Transfer for Semantic Scene Segmentation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Antonio Torralba,et al.  Nonparametric scene parsing: Label transfer via dense scene alignment , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Chi-Keung Tang,et al.  Local color transfer via probabilistic segmentation by expectation-maximization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Gabriel Peyré,et al.  Wasserstein barycentric coordinates , 2016, ACM Trans. Graph..

[10]  Raimondo Schettini,et al.  Color transfer using semantic image annotation , 2011, Electronic Imaging.

[11]  A.C. Kokaram,et al.  N-dimensional probability density function transfer and its application to color transfer , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[12]  Thomas G. Moher.  Videocassette Course Development Using Microcomputer Graphics , 1985, IEEE Computer Graphics and Applications.

[13]  Kwan H. Lee,et al.  Local color transfer between images using dominant colors , 2013, J. Electronic Imaging.

[14]  Xing Mei,et al.  Content‐Based Colour Transfer , 2013, Comput. Graph. Forum.

[15]  Masayuki Nakajima,et al.  Example-Based Color Stylization of Images , 2005, TAP.

[16]  Russell Zaretzki,et al.  Emotion transfer for images based on color combinations , 2013, ArXiv.

[17]  Keiji Uchikawa,et al.  Example-based color stylization based on categorical perception , 2004, APGV '04.

[18]  Hui Huang,et al.  Image recoloring using geodesic distance based color harmonization , 2015, Computational Visual Media.

[19]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[20]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Naila Murray,et al.  Toward automatic and flexible concept transfer , 2012, Comput. Graph..

[22]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[23]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[24]  Erik Reinhard,et al.  Progressive histogram reshaping for creative color transfer and tone reproduction , 2010, NPAR.

[25]  Yizhou Yu,et al.  Data-driven image color theme enhancement , 2010, ACM Trans. Graph..

[26]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[27]  Stephen DiVerdi,et al.  Palette-based photo recoloring , 2015, ACM Trans. Graph..

[28]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[29]  Lizhuang Ma,et al.  Gradient‐Preserving Color Transfer , 2009, Comput. Graph. Forum.

[30]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[31]  Fabio Pellacini,et al.  User‐Controllable Color Transfer , 2010, Comput. Graph. Forum.

[32]  Bing-Yu Chen,et al.  Example‐based Multiple Local Color Transfer by Strokes , 2008, Comput. Graph. Forum.

[33]  Gabriela Csurka,et al.  Building look & feel concept models from color combinations , 2011, The Visual Computer.

[34]  J. Russell.  A circumplex model of affect. , 1980 .

[35]  Lianhong Cai,et al.  Affective Image Colorization , 2012, Journal of Computer Science and Technology.