Image Retargeting by Content-Aware Synthesis

Real-world images usually contain vivid contents and rich textural details, which will complicate the manipulation on them. In this paper, we design a new framework based on content-aware synthesis to enhance content-aware image retargeting. By detecting the textural regions in an image, the textural image content can be synthesized rather than simply distorted or cropped. This method enables the manipulation of textural & non-textural regions with different strategy since they have different natures. We propose to retarget the textural regions by content-aware synthesis and non-textural regions by fast multi-operators. To achieve practical retargeting applications for general images, we develop an automatic and fast texture detection method that can detect multiple disjoint textural regions. We adjust the saliency of the image according to the features of the textural regions. To validate the proposed method, comparisons with state-of-the-art image targeting techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method.

[1]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Irfan Essa,et al.  Texture optimization for example-based synthesis , 2005, SIGGRAPH 2005.

[4]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Kun Zhou,et al.  Fast example-based surface texture synthesis via discrete optimization , 2006, The Visual Computer.

[6]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Chi-Keung Tang,et al.  KNN Matting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Tien-Tsin Wong,et al.  Resizing by symmetry-summarization , 2010, ACM Trans. Graph..

[9]  Sylvain Lefebvre,et al.  State of the Art in Example-based Texture Synthesis , 2009, Eurographics.

[10]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  C.-C. Jay Kuo,et al.  Region-adaptive texture-aware image resizing , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[14]  Ralph R. Martin,et al.  Shrinkability Maps for Content‐Aware Video Resizing , 2008, Comput. Graph. Forum.

[15]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Shai Avidan,et al.  Stereo Seam Carving a Geometrically Consistent Approach , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, SIGGRAPH 2008.

[18]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[19]  Jan-Olof Eklundh,et al.  Real-time texture detection using the LU-transform , 2006 .

[20]  Weiming Dong,et al.  Perspective-aware texture analysis and synthesis , 2008, The Visual Computer.

[21]  S. Avidan,et al.  Multi-operator media retargeting , 2009, SIGGRAPH 2009.

[22]  Li Xu,et al.  Structure extraction from texture via relative total variation , 2012, ACM Trans. Graph..

[23]  Paul F. Whelan,et al.  Image segmentation based on the integration of colour-texture descriptors - A review , 2011, Pattern Recognit..

[24]  Weiming Dong,et al.  Optimized image resizing using seam carving and scaling , 2009, SIGGRAPH 2009.

[25]  Narendra Ahuja,et al.  Texel-based texture segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[26]  Yung-Yu Chuang,et al.  A line-structure-preserving approach to image resizing , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Tong-Yee Lee,et al.  Summarization-Based Image Resizing by Intelligent Object Carving. , 2013, IEEE transactions on visualization and computer graphics.

[28]  O. Sorkine,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH 2008.

[29]  Markus Gross,et al.  A system for retargeting of streaming video , 2009, SIGGRAPH 2009.

[30]  Wonjun Kim,et al.  A Texture-Aware Salient Edge Model for Image Retargeting , 2011, IEEE Signal Processing Letters.

[31]  Yong-Jin Liu,et al.  Optimal-Scaling-Factor Assignment for Patch-wise Image Retargeting , 2013, IEEE Computer Graphics and Applications.

[32]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[33]  Xiao Liu,et al.  Probabilistic Graphlet Transfer for Photo Cropping , 2013, IEEE Transactions on Image Processing.

[34]  Chao-Hung Lin,et al.  Object-Coherence Warping for Stereoscopic Image Retargeting , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Yi Yang,et al.  Weakly Supervised Photo Cropping , 2014, IEEE Transactions on Multimedia.

[36]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Jian Shi,et al.  Image Retargeting Using Mesh Parametrization , 2009, IEEE Transactions on Multimedia.

[38]  Kun Zhou,et al.  Inverse texture synthesis , 2008, ACM Trans. Graph..

[39]  Patrick Pérez,et al.  Geodesic image and video editing , 2010, TOGS.

[40]  Siglas de Palabras B.U.J. , 2013 .

[41]  Aykut Erdem,et al.  Structure-preserving image smoothing via region covariances , 2013, ACM Trans. Graph..

[42]  Aljoscha Smolic,et al.  Finite Element Image Warping , 2013, Comput. Graph. Forum.

[43]  Ruth Bergman,et al.  Detection of Textured Areas in Images Using a Disorganization Indicator Based on Component Counts , 2007 .

[44]  Chao-Hung Lin,et al.  Patch-Based Image Warping for Content-Aware Retargeting , 2013, IEEE Transactions on Multimedia.

[45]  Olga Sorkine-Hornung,et al.  Robust Image Retargeting via Axis‐Aligned Deformation , 2012, Comput. Graph. Forum.

[46]  Sylvain Lefebvre,et al.  Appearance-space texture synthesis , 2006, ACM Trans. Graph..

[47]  Daniel Cohen-Or,et al.  Feature-aware texturing , 2006, EGSR '06.

[48]  Stephen Lin,et al.  Learning the Change for Automatic Image Cropping , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  William T. Freeman,et al.  The patch transform and its applications to image editing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Olga Sorkine-Hornung,et al.  A comparative study of image retargeting , 2010, ACM Trans. Graph..

[52]  Ariel Shamir,et al.  Cropping Scaling Seam carving Warping Multi-operator , 2009 .

[53]  Weiming Dong,et al.  Fast Multi-Operator Image Resizing and Evaluation , 2012, Journal of Computer Science and Technology.

[54]  A. Ravishankar Rao,et al.  Identifying High Level Features of Texture Perception , 1993, CVGIP Graph. Model. Image Process..

[55]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[56]  Jean-Michel Morel,et al.  Fast Cartoon + Texture Image Filters , 2010, IEEE Transactions on Image Processing.

[57]  P. Heng,et al.  Resizing by symmetry-summarization , 2010, SIGGRAPH 2010.

[58]  Kup-Sze Choi,et al.  Fast Gabor texture feature extraction with separable filters using GPU , 2013, Journal of Real-Time Image Processing.