Importance filtering for image retargeting

Content-aware image retargeting has attracted a lot of interests recently. The key and most challenging issue for this task is how to balance the tradeoff between preserving the important contents and minimizing the visual distortions on the consistency of the image structure. In this paper we present a novel filtering-based technique to tackle this issue, called ”importance filtering”. Specifically, we first filter the image saliency guided by the image itself to achieve a structure-consistent importance map. We then use the pixel importance as the key constraint to compute the gradient map of pixel shifts from the original resolution to the target. Finally, we integrate the shift gradient across the image using a weighted filter to construct a smooth shift map and render the target image. The weight is again controlled by the pixel importance. The two filtering processes enforce to maintain the structural consistency and yet preserve the important contents in the target image. Furthermore, the simple nature of filter operations allows highly efficient implementation for real-time applications and easy extension to video retargeting, as the structural constraints from the original image naturally convey the temporal coherence between frames. The effectiveness and efficiency of our importance filtering algorithm are confirmed in extensive experiments.

[1]  Pradeep Sen,et al.  Video Carving , 2008, Eurographics.

[2]  Olga Sorkine-Hornung,et al.  A comparative study of image retargeting , 2010, ACM Trans. Graph..

[3]  Ariel Shamir,et al.  Cropping Scaling Seam carving Warping Multi-operator , 2009 .

[4]  Shang-Hong Lai,et al.  Fast structure-preserving image retargeting , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  O. Sorkine,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH 2008.

[6]  Markus Gross,et al.  A system for retargeting of streaming video , 2009, SIGGRAPH 2009.

[7]  Ralph R. Martin,et al.  A Shape‐Preserving Approach to Image Resizing , 2009, Comput. Graph. Forum.

[8]  Jing Xiao,et al.  Bi-affinity Filter: A Bilateral Type Filter for Color Images , 2010, ECCV Workshops.

[9]  Deepu Rajan,et al.  Hybrid shift map for video retargeting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[11]  Mei Han,et al.  Discontinuous seam-carving for video retargeting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[13]  James W. Davis,et al.  An appearance-based representation of action , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[14]  David Salesin,et al.  Gaze-based interaction for semi-automatic photo cropping , 2006, CHI.

[15]  Duan-Yu Chen,et al.  Content-Aware Video Seam Carving Based on Bag of Visual Cubes , 2010, 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[16]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, SIGGRAPH 2008.

[17]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Craig Gotsman,et al.  Energy‐Based Image Deformation , 2009, Comput. Graph. Forum.

[19]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  H. Seidel,et al.  Motion-aware temporal coherence for video resizing , 2009, SIGGRAPH 2009.

[22]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[23]  Daniel Cohen-Or,et al.  Feature-aware texturing , 2006, EGSR '06.

[24]  Benjamin B. Bederson,et al.  Automatic thumbnail cropping and its effectiveness , 2003, UIST '03.

[25]  ShamirAriel,et al.  Seam carving for content-aware image resizing , 2007 .

[26]  Alvaro Soto,et al.  Human detection using a mobile platform and novel features derived from a visual saliency mechanism , 2010, Image Vis. Comput..