Aesthetic Image Enhancement by Dependence-Aware Object Recomposition

This paper proposes an image-enhancement method to optimize photograph composition by rearranging foreground objects in the photograph. To adjust objects' positions while keeping the original scene content, we first perform a novel structure dependence analysis on the image to obtain the dependencies between all background regions. To determine the optimal positions for foreground objects, we formulate an optimization problem based on widely used heuristics for aesthetically pleasing pictures. Semantic relations between foreground objects are also taken into account during optimization. The final output is produced by moving foreground objects, together with their dependent regions, to optimal positions. The results show that our approach can effectively optimize photographs with single or multiple foreground objects without compromising the original photograph content.

[1]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[2]  Yongwei Nie,et al.  Fast Exact Nearest Patch Matching for Patch-Based Image Editing and Processing , 2011, IEEE Transactions on Visualization and Computer Graphics.

[3]  David Salesin,et al.  Gaze-based interaction for semi-automatic photo cropping , 2006, CHI.

[4]  Maneesh Agrawala,et al.  Interactive furniture layout using interior design guidelines , 2011, SIGGRAPH 2011.

[5]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[6]  Brian L. Evans,et al.  In-Camera Automation of Photographic Composition Rules , 2007, IEEE Transactions on Image Processing.

[7]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[8]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[10]  Yun Zhang,et al.  Environment-Sensitive cloning in images , 2011, The Visual Computer.

[11]  Ligang Liu,et al.  Realtime Aesthetic Image Retargeting , 2010, CAe.

[12]  Qunsheng Peng,et al.  Robust image segmentation against complex color distribution , 2011, The Visual Computer.

[13]  Zeev Farbman,et al.  Coordinates for instant image cloning , 2009, ACM Trans. Graph..

[14]  王文成,et al.  Free Appearance-Editing with Improved Poisson Image Cloning , 2011 .

[15]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[16]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, ACM Trans. Graph..

[17]  Tsuhan Chen,et al.  Towards aesthetics: a photo quality assessment and photo selection system , 2010, ACM Multimedia.

[18]  W. Chu Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[19]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[20]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[21]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Daniel Cohen-Or,et al.  Combining color and depth for enhanced image segmentation and retargeting , 2011, The Visual Computer.

[23]  Yoichi Sato,et al.  Sensation-based photo cropping , 2009, ACM Multimedia.

[24]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[25]  L. Ou,et al.  A colour harmony model for two-colour combinations , 2006 .

[26]  Hua Huang,et al.  Arcimboldo-like collage using internet images , 2011, ACM Trans. Graph..

[27]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[28]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[29]  Xun Wang,et al.  Adaptive tone-preserved image detail enhancement , 2012, The Visual Computer.

[30]  Tiejun Huang,et al.  Salient region detection and segmentation for general object recognition and image understanding , 2011, Science China Information Sciences.

[31]  Yanwen Guo,et al.  Improving Photo Composition Elegantly: Considering Image Similarity During Composition Optimization , 2012, Comput. Graph. Forum.

[32]  R. Arnheim Art and visual perception: A psychology of the creative eye, New version , 1955 .

[33]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[34]  Daniel Cohen-Or,et al.  Optimizing Photo Composition , 2010, Comput. Graph. Forum.

[35]  Lianhong Cai,et al.  Affective Image Colorization , 2012, Journal of Computer Science and Technology.

[36]  H. Leder,et al.  A model of aesthetic appreciation and aesthetic judgments. , 2004, British journal of psychology.

[37]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[38]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Julie Dorsey,et al.  Understanding and improving the realism of image composites , 2012, ACM Trans. Graph..

[40]  Dong Wang,et al.  Saliency-driven scaling optimization for image retargeting , 2011, The Visual Computer.

[41]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[42]  Shi-Min Hu,et al.  Popup: automatic paper architectures from 3D models , 2010, SIGGRAPH 2010.

[43]  Patrick Olivier,et al.  Advanced Composition in Virtual Camera Control , 2011, Smart Graphics.

[44]  Robert M. Haralick,et al.  Feature normalization and likelihood-based similarity measures for image retrieval , 2001, Pattern Recognit. Lett..

[45]  Michael Freeman,et al.  The Photographer's Eye: Composition and Design for Better Digital Photos , 2007 .

[46]  S. B. Kang,et al.  Image deblurring using inertial measurement sensors , 2010, SIGGRAPH 2010.

[47]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[48]  Anton Osokin,et al.  Fast Approximate Energy Minimization with Label Costs , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Kun Zhou,et al.  Interactive images , 2012, ACM Trans. Graph..

[50]  Mubarak Shah,et al.  A holistic approach to aesthetic enhancement of photographs , 2011, TOMCCAP.