Saliency for image manipulation

Every picture tells a story. In photography, the story is portrayed by a composition of objects, commonly referred to as the subjects of the piece. Were we to remove these objects, the story would be lost. When manipulating images, either for artistic rendering or cropping, it is crucial that the story of the piece remains intact. As a result, the knowledge of the location of these prominent objects is essential. We propose an approach for saliency detection that combines previously suggested patch distinctness with an object probability map. The object probability map infers the most probable locations of the subjects of the photograph according to highly distinct salient cues. The benefits of the proposed approach are demonstrated through state-of-the-art results on common data sets. We further show the benefit of our method in various manipulations of real-world photographs while preserving their meaning.

[1]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[4]  William Prinzmetal,et al.  Visual Feature Integration in a World of Objects , 1995 .

[5]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[6]  Pascal Fua,et al.  Image summaries using database saliency , 2009, SIGGRAPH ASIA '09.

[7]  John Collomosse,et al.  Painterly rendering using image salience , 2002, Proceedings 20th Eurographics UK Conference.

[8]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Ralph R. Martin,et al.  A Shape‐Preserving Approach to Image Resizing , 2009, Comput. Graph. Forum.

[10]  Song-Chun Zhu,et al.  From image parsing to painterly rendering , 2009, TOGS.

[11]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[13]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[14]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[15]  Ariel Shamir,et al.  Cropping Scaling Seam carving Warping Multi-operator , 2009 .

[16]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[17]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[19]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[20]  Tomer Carmel,et al.  Perceptual objects capture attention , 2009, Vision Research.

[21]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.