Image aesthetics enhancement using composition-based saliency detection

Visual saliency detection and segmentation are widely used in many applications in image processing and computer vision. However, existing saliency detection methods have not fully taken the spatial information of salient regions into account. Inspired by the basic photographic composition rules, we present a novel saliency detection method, which utilizes the knowledge of photographic composition as priors to improve the saliency detection results. Moreover, an online parameter selection method is proposed when utilizing GrabCut to achieve the saliency segmentation result. Besides, to test the applicability of our method, we present a novel post-processing framework for the photographs to be more artistic. The salient region and depth map are firstly computed. The salient region keeps its sharpness, while other parts in the photograph get blurred based on the depth map. To our best knowledge, this is a novel image-based attempt to enhance aesthetics by post-processing a photograph via realistic blurring. We test our method on the 1,000 benchmark test images and dataset MSRA. Extensive experimental results show the applicability and effectiveness of our method.

[1]  Shi-Min Hu,et al.  SalientShape: group saliency in image collections , 2013, The Visual Computer.

[2]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[3]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[4]  Masaaki Ikehara,et al.  HMM-based surface reconstruction from single images , 2002, Proceedings. International Conference on Image Processing.

[5]  Stephen Marshall,et al.  Threshold decomposition driven adaptive morphological filter for image sharpening , 2007, VISAPP.

[6]  Nitin Sampat,et al.  Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications , 2000 .

[7]  N. Sebe,et al.  Facial Expression Recognition: A Fully Integrated Approach , 2007, 14th International Conference of Image Analysis and Processing - Workshops (ICIAPW 2007).

[8]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[9]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[10]  Meng Wang,et al.  Dynamic captioning: video accessibility enhancement for hearing impairment , 2010, ACM Multimedia.

[11]  Gabriele Peters,et al.  Aesthetic Primitives of Images for Visualization , 2007, 2007 11th International Conference Information Visualization (IV '07).

[12]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[13]  Andrew B. Watson,et al.  Toward a perceptual video-quality metric , 1998, Electronic Imaging.

[14]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[15]  Bo Peng,et al.  Parameter Selection for Graph Cut Based Image Segmentation , 2008, BMVC.

[16]  James Ze Wang,et al.  Learning the consensus on visual quality for next-generation image management , 2007, ACM Multimedia.

[17]  Timo Schairer,et al.  Realistic Depth Blur for Images with Range Data , 2009, Dyn3D.

[18]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ashutosh Saxena,et al.  3-D Depth Reconstruction from a Single Still Image , 2007, International Journal of Computer Vision.

[20]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[21]  S. Zeki,et al.  A direct demonstration of perceptual asynchrony in vision , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[22]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[23]  Daniel Cohen-Or,et al.  Optimizing Photo Composition , 2010, Comput. Graph. Forum.

[24]  Wei-Ying Ma,et al.  Auto cropping for digital photographs , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[25]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[26]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[27]  Meng Wang,et al.  Video accessibility enhancement for hearing-impaired users , 2011, TOMCCAP.

[28]  Ashutosh Saxena,et al.  Learning Depth from Single Monocular Images , 2005, NIPS.

[29]  Narendra Ahuja,et al.  Performance Analysis of Stereo, Vergence, and Focus as Depth Cues for Active Vision , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  W. Chu.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[31]  Nicu Sebe,et al.  Sonify your face: facial expressions for sound generation , 2010, ACM Multimedia.

[32]  Xiaochun Cao,et al.  Visual saliency detection based on photographic composition , 2013, ICIMCS '13.

[33]  Meng Wang,et al.  Movie2Comics: Towards a Lively Video Content Presentation , 2012, IEEE Transactions on Multimedia.

[34]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[35]  Jitendra Malik,et al.  Computing Local Surface Orientation and Shape from Texture for Curved Surfaces , 1997, International Journal of Computer Vision.

[36]  Marcel J. T. Reinders,et al.  Image sharpening by morphological filtering , 2000, Pattern Recognit..

[37]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Frédo Durand,et al.  Defocus video matting , 2005, SIGGRAPH 2005.

[39]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[40]  Atsuto Maki,et al.  Geotensity: Combining Motion and Lighting for 3D Surface Reconstruction , 2004, International Journal of Computer Vision.

[41]  Murali Subbarao,et al.  Focused image recovery from two defocused images recorded with different camera settings , 1995, IEEE Transactions on Image Processing.

[42]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[44]  James Ze Wang,et al.  Algorithmic inferencing of aesthetics and emotion in natural images: An exposition , 2008, 2008 15th IEEE International Conference on Image Processing.

[45]  Sam Kavusi,et al.  Computationally efficient algorithm for multifocus image reconstruction , 2003, IS&T/SPIE Electronic Imaging.