Image retargeting using nonparametric semantic segmentation

In this paper, a new full-automatic approach to content aware image retargeting is proposed. Most image retargeting approaches does not incorporate content information and only use local appearance information. However, there are some approaches which use high level information such as saliency regions, objects mask and depth information. Such methods do not use semantic labelling for each object. In this paper, object masks as well as their semantic class labels are used to propose a new approach to image retargeting. To do so, semantic segmentation of image is provided. Hence, a nonparametric approach to semantic segmentation is employed which is fast with no need to any learning model. This makes it simple and applicable to any dataset. To evaluate the proposed approach, besides presenting visual examples, we performed a set of subjective evaluations too. The obtained results show that our method outperforms other retargeting approaches.

[1]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Ramesh Raskar,et al.  Automatic image retargeting , 2004, SIGGRAPH '04.

[3]  Stephen Gould,et al.  PatchMatchGraph: Building a Graph of Dense Patch Correspondences for Label Transfer , 2012, ECCV.

[4]  Craig Gotsman,et al.  Energy‐Based Image Deformation , 2009, Comput. Graph. Forum.

[5]  Sanja Fidler,et al.  Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Joost van de Weijer,et al.  Harmony potentials for joint classification and segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[9]  Antonio Torralba,et al.  Nonparametric Scene Parsing via Label Transfer , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Ariel Shamir,et al.  Cropping Scaling Seam carving Warping Multi-operator , 2009 .

[11]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[12]  Natasha Gelfand,et al.  A survey of image retargeting techniques , 2010, Optical Engineering + Applications.

[13]  Olga Sorkine-Hornung,et al.  Visual media retargeting , 2009, SIGGRAPH ASIA Courses.

[14]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Luc Van Gool,et al.  Scene Carving: Scene Consistent Image Retargeting , 2010, ECCV.

[16]  Bernd Grtner,et al.  Approximation Algorithms and Semidefinite Programming , 2012 .

[17]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Heesoo Myeong,et al.  Tensor-Based High-Order Semantic Relation Transfer for Semantic Scene Segmentation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Shadrokh Samavi,et al.  A new fast approach to nonparametric scene parsing , 2014, Pattern Recognit. Lett..

[20]  O. Sorkine,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH 2008.

[21]  Larry S. Davis,et al.  Learning What and How of Contextual Models for Scene Labeling , 2010, ECCV.

[22]  Yu Gong,et al.  Semantic aware sport image resizing jointly using seam carving and warping , 2012, Multimedia Tools and Applications.

[23]  Sabine Süsstrunk,et al.  Saliency detection for content-aware image resizing , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[24]  Cordelia Schmid,et al.  Coloring Local Feature Extraction , 2006, ECCV.

[25]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Stephen P. Boyd,et al.  The CVX Users' Guide , 2015 .

[27]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.