No-reference Harmony-Guided Quality Assessment

Color harmony of simple color patterns has been widely studied for color design. Rules defined then by psychological experiments have been applied to derive image aesthetic scores, or to re-colorize pictures. But what is harmonious or not in an image? What can the human eye perceive disharmonious? Extensive research has been done in the context of quality assessment to define what is visible or not in images and videos. Techniques based on human visual system models use signal masking to define visibility thresholds. Based on results in both fields, we present a harmony quality assessment method to assess what is harmonious or not in an image. Color rules are used to detect what part of images are disharmonious, and visual masking is applied to estimate to what extent an image area can be perceived disharmonious. The output perceptual harmony quality map and scores can be used in a photo editing framework to guide the user getting the best artistic effects. Results show that the harmony maps reflect what a user perceives and that the score is correlated to the artistic intent.

[1]  Sheila S. Hemami,et al.  A metric for continuous quality evaluation of compressed video with severe distortions , 2004, Signal Process. Image Commun..

[2]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[3]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[4]  Jun Li,et al.  Structure and Hue Similarity for Color Image Quality Assessment , 2009, 2009 International Conference on Electronic Computer Technology.

[5]  Patrick Le Callet,et al.  On the performance of human visual system based image quality assessment metric using wavelet domain , 2008, Electronic Imaging.

[6]  M. Nadenau.  Integration of human color vision models into high quality image compression , 2000 .

[7]  Tsuhan Chen,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[8]  Andrew B. Watson,et al.  Image quality and entropy masking , 1997, Electronic Imaging.

[9]  J. Itten.  The art of color : the subjective experience and objective rationale of color , 1973 .

[10]  Pierre Hellier,et al.  Saliency-Guided Consistent Color Harmonization , 2013, CCIW.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Daniel Cohen-Or,et al.  Color harmonization , 2006, ACM Trans. Graph..

[13]  Swapna Devi,et al.  A New Method for Color Image Quality Assessment , 2011 .

[14]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[15]  Chu-Song Chen,et al.  Video aesthetic quality assessment by combining semantically independent and dependent features , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Nuria Oliver,et al.  Towards Computational Models of the Visual Aesthetic Appeal of Consumer Videos , 2010, ECCV.

[17]  Scott Daly,et al.  Digital Images and Human Vision , 1993 .

[18]  Andrew B. Watson,et al.  Digital images and human vision , 1993 .

[19]  Patrick Le Callet,et al.  Which semi-local visual masking model forwavelet based image quality metric? , 2008, 2008 15th IEEE International Conference on Image Processing.

[20]  D. Spencer,et al.  Geometric Formulation of Classical Color Harmony , 1944 .

[21]  Masashi Nishiyama,et al.  Aesthetic quality classification of photographs based on color harmony , 2011, CVPR 2011.

[22]  Carman Neustaedter,et al.  Image harmony for consumer images , 2008, 2008 15th IEEE International Conference on Image Processing.

[23]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[24]  Stefan Winkler,et al.  Perceptual distortion metric for digital color video , 1999, Electronic Imaging.