A framework for photo-quality assessment and enhancement based on visual aesthetics

We present an interactive application that enables users to improve the visual aesthetics of their digital photographs using spatial recomposition. Unlike earlier work that focuses either on photo quality assessment or interactive tools for photo editing, we enable the user to make informed decisions about improving the composition of a photograph and to implement them in a single framework. Specifically, the user interactively selects a foreground object and the system presents recommendations for where it can be moved in a manner that optimizes a learned aesthetic metric while obeying semantic constraints. For photographic compositions that lack a distinct foreground object, our tool provides the user with cropping or expanding recommendations that improve its aesthetic quality. We learn a support vector regression model for capturing image aesthetics from user data and seek to optimize this metric during recomposition. Rather than prescribing a fully-automated solution, we allow user-guided object segmentation and inpainting to ensure that the final photograph matches the user's criteria. Our approach achieves 86% accuracy in predicting the attractiveness of unrated images, when compared to their respective human rankings. Additionally, 73% of the images recomposited using our tool are ranked more attractive than their original counterparts by human raters.

[1]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[2]  James Ze Wang,et al.  Learning the consensus on visual quality for next-generation image management , 2007, ACM Multimedia.

[3]  Brian L. Evans,et al.  Unsupervised automation of photographic composition rules in digital still cameras , 2004, IS&T/SPIE Electronic Imaging.

[4]  Miska M. Hannuksela,et al.  Perceptual quality assessment based on visual attention analysis , 2009, ACM Multimedia.

[5]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[6]  Ajmal S. Mian,et al.  A Hybrid Image Quality Measure for Automatic Image Quality Assessment , 2009, SCIA.

[7]  Dani Lischinski,et al.  Data-driven enhancement of facial attractiveness , 2008, ACM Trans. Graph..

[8]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[9]  Yoichi Sato,et al.  Sensation-based photo cropping , 2009, ACM Multimedia.

[10]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[12]  Richard Szeliski,et al.  Automatic Estimation and Removal of Noise from a Single Image , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  The golden ratio and aesthetics , 2006 .

[14]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[15]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[17]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[18]  William T. Freeman,et al.  The patch transform and its applications to image editing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Mubarak Shah,et al.  Region Completion in a Single Image , 2004, Eurographics.

[20]  Rongrong Ji,et al.  Photo assessment based on computational visual attention model , 2009, ACM Multimedia.

[21]  Jiebo Luo,et al.  Bayesian fusion of camera metadata cues in semantic scene classification , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[22]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.