Multi-operator retargeting based on perceptual structural similarity

We propose a new multi-operator retargeting algorithm by using three resizing operators of seam carving, scaling, and cropping iteratively. To determine which operator should be used at each iteration, we adopt structural similarity (SSIM) to evaluate the similarity between the original and retargeted images for the dynamic programming. Since the sizes of original and retargeted images are different, SIFT flow is used for dense correspondence between the original and retargeted images for similarity evaluation. Additionally, visual saliency is used to weight SSIM results based on the characteristics of the Human Visual System (HVS). Experimental results on a public image retargeting database show the promising performance of the proposed multi-operator retargeting algorithm.

[1]  Weiming Dong,et al.  Optimized image resizing using seam carving and scaling , 2009, SIGGRAPH 2009.

[2]  Chang-Su Kim,et al.  Adaptive image and video retargeting technique based on Fourier analysis , 2009, CVPR.

[3]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Mei Han,et al.  Discontinuous seam-carving for video retargeting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Olga Sorkine-Hornung,et al.  A comparative study of image retargeting , 2010, ACM Trans. Graph..

[6]  Kai Zeng,et al.  Objective Quality Assessment for Image Retargeting Based on Structural Similarity , 2014, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[7]  Ariel Shamir,et al.  Cropping Scaling Seam carving Warping Multi-operator , 2009 .

[8]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[9]  O. Sorkine,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH 2008.

[10]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.

[11]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[12]  Jian Shi,et al.  Image Retargeting Using Mesh Parametrization , 2009, IEEE Transactions on Multimedia.

[13]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, SIGGRAPH 2008.

[14]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[15]  William T. Freeman,et al.  The patch transform and its applications to image editing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[17]  Natasha Gelfand,et al.  A survey of image retargeting techniques , 2010, Optical Engineering + Applications.

[18]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[19]  Ligang Liu,et al.  Nonhomogeneous scaling optimization for realtime image resizing , 2010, The Visual Computer.

[20]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[21]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Diego Gutierrez,et al.  Using eye-tracking to assess different image retargeting methods , 2011, APGV '11.

[23]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[24]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.