Learning Natural Colors for Image Recoloring

We present a data‐driven method for automatically recoloring a photo to enhance its appearance or change a viewer's emotional response to it. A compact representation called a RegionNet summarizes color and geometric features of image regions, and geometric relationships between them. Correlations between color property distributions and geometric features of regions are learned from a database of well‐colored photos. A probabilistic factor graph model is used to summarize distributions of color properties and generate an overall probability distribution for color suggestions. Given a new input image, we can generate multiple recolored results which unlike previous automatic results, are both natural and artistic, and compatible with their spatial arrangements.

[1]  Xing Mei,et al.  Content‐Based Colour Transfer , 2013, Comput. Graph. Forum.

[2]  Zeev Farbman,et al.  Interactive local adjustment of tonal values , 2006, ACM Trans. Graph..

[3]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jieping Ye,et al.  A framework for interactive image color editing , 2012, The Visual Computer.

[5]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[6]  Erik Reinhard,et al.  A Survey of Color Mapping and its Applications , 2014, Eurographics.

[7]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[8]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Chun Chen,et al.  Data-driven image color theme enhancement , 2010, SIGGRAPH 2010.

[10]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shi-Min Hu,et al.  Efficient affinity-based edit propagation using K-D tree , 2009, SIGGRAPH 2009.

[12]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..

[13]  Yi Wang,et al.  Vector morphological operators in HSV color space , 2011, Science China Information Sciences.

[14]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Ralph R. Martin,et al.  Internet visual media processing: a survey with graphics and vision applications , 2013, The Visual Computer.

[16]  Klaus Mueller,et al.  A Data-Driven Approach to Hue-Preserving Color-Blending , 2012, IEEE Transactions on Visualization and Computer Graphics.

[17]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[18]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[19]  Ralph R. Martin,et al.  PatchNet: a patch-based image representation for interactive library-driven image editing , 2013, ACM Trans. Graph..

[20]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[21]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Lianhong Cai,et al.  Affective image adjustment with a single word , 2013, The Visual Computer.

[23]  Lianhong Cai,et al.  Affective Image Colorization , 2012, Journal of Computer Science and Technology.

[24]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[25]  Natasha Gelfand,et al.  TouchTone: Interactive Local Image Adjustment Using Point‐and‐Swipe , 2010, Comput. Graph. Forum.

[26]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[27]  Stephen Lin,et al.  Semantic colorization with internet images , 2011, ACM Trans. Graph..

[28]  Sylvain Paris,et al.  Example-based video color grading , 2013, ACM Trans. Graph..

[29]  Carsten Rother,et al.  Clustering appearance and shape by learning jigsaws , 2006, NIPS.

[30]  Bernhard Schölkopf,et al.  Automatic Image Colorization Via Multimodal Predictions , 2008, ECCV.

[31]  Chi-Keung Tang,et al.  Local color transfer via probabilistic segmentation by expectation-maximization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Shi-Min Hu,et al.  Instant Propagation of Sparse Edits on Images and Videos , 2010, Comput. Graph. Forum.

[33]  Pat Hanrahan,et al.  Probabilistic color-by-numbers , 2013, ACM Trans. Graph..

[34]  Aaron Hertzmann,et al.  Color compatibility from large datasets , 2011, ACM Trans. Graph..

[35]  Deepu Rajan,et al.  Image colorization using similar images , 2012, ACM Multimedia.