Multi-operator image retargeting with automatic integration of direct and indirect seam carving

Multi-operator image resizing can preserve important objects and structure in an image by combining multiple image resizing operators. However, traditional multi-operator methods do not take both horizontal and vertical content-aware resizing potential into consideration, which essentially leads to squeeze/stretch effect in the resultant images. In this paper, we propose a new multi-operator scheme that addresses aforementioned issue by integrating direct and indirect seam carving. Compared with previous methods, the proposed scheme remarkably reduces the cost of deciding when to change operators, by employing a newly defined image artifact measure. Furthermore, we propose a novel seam carving enhancement, named ACcumulated Energy Seam Carving (ACESC), as a basic operator to improve global structure preservation. By combining horizontal and vertical seam carving, our scheme preserves the shapes of important objects well. We present typical results to demonstrate the effectiveness of our method. User study shows that our method has high user preference.

[1]  Ariel Shamir,et al.  A comparative study of image retargeting , 2010, SIGGRAPH 2010.

[2]  Liang-Tien Chia,et al.  Robust subspace analysis for detecting visual attention regions in images , 2005, MULTIMEDIA '05.

[3]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  David Salesin,et al.  Gaze-based interaction for semi-automatic photo cropping , 2006, CHI.

[5]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, ACM Trans. Graph..

[6]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[7]  Tibor Bosse,et al.  A Cognitive Model for Visual Attention and Its Application , 2006, 2006 IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[8]  Mei Han,et al.  Discontinuous seam-carving for video retargeting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[10]  Jian Shi,et al.  Image Retargeting Using Mesh Parametrization , 2009, IEEE Transactions on Multimedia.

[11]  Xing Xie,et al.  Automatic browsing of large pictures on mobile devices , 2003, MULTIMEDIA '03.

[12]  Natasha Gelfand,et al.  A survey of image retargeting techniques , 2010, Optical Engineering + Applications.

[13]  Craig Gotsman,et al.  Energy‐Based Image Deformation , 2009, Comput. Graph. Forum.

[14]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[17]  Hermann Ney,et al.  Pan, zoom, scan — Time-coherent, trained automatic video cropping , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Mark Ollila,et al.  Eurographics Symposium on Rendering , 2004 .

[19]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[20]  Benjamin B. Bederson,et al.  Automatic thumbnail cropping and its effectiveness , 2003, UIST '03.

[21]  Weiming Dong,et al.  Optimized image resizing using seam carving and scaling , 2009, SIGGRAPH 2009.

[22]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[23]  Michael Gleicher,et al.  Automatic image retargeting with fisheye-view warping , 2005, UIST.

[24]  S. Avidan,et al.  Multi-operator media retargeting , 2009, SIGGRAPH 2009.

[25]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  O. Sorkine,et al.  Motion-based video retargeting with optimized crop-and-warp , 2010, ACM Trans. Graph..

[28]  Ralph R. Martin,et al.  A Shape‐Preserving Approach to Image Resizing , 2009, Comput. Graph. Forum.

[29]  Ralph R. Martin,et al.  Shrinkability Maps for Content‐Aware Video Resizing , 2008, Comput. Graph. Forum.

[30]  Markus H. Gross,et al.  A system for retargeting of streaming video , 2009, ACM Trans. Graph..

[31]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, SIGGRAPH 2008.

[32]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[33]  Raimondo Schettini,et al.  Self-Adaptive Image Cropping for Small Displays , 2007, IEEE Transactions on Consumer Electronics.

[34]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[35]  Daniel Cohen-Or,et al.  Feature-aware texturing , 2006, EGSR '06.

[36]  Olga Sorkine-Hornung,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH Asia '08.

[37]  Xing Xie,et al.  A visual attention model for adapting images on small displays , 2003, Multimedia Systems.