Conditional Random Field based salient proposal set generation and its application in content aware seam carving

Abstract Most of the existing generic object localization algorithms usually give the plausible object locations without taking into consideration the saliency ordering of the proposal set. This paper presents a novel object proposal generation which ranks the key objects according to their saliency score in the proposal pool. First, we formulate a Bayesian framework for generating a probabilistic edgemap which is used to assign a saliency value to the edgelets. A conditional random field is then learnt for edge-labeling by effectively combining the edge features with the relative spatial layout of the edge segments. Lastly, we propose an objectness score for the generated proposal set by analyzing the salient object edge density completely lying within the candidate boxes. Extensive experiments on the benchmark PASCAL VOC 2007 and 2012 datasets demonstrate that the proposed method provides competitive performance against popular generic object detection techniques while using fewer number of proposals. Additionally, we demonstrate the applicability of the generated proposal set for content aware image retargeting.

[1]  Brejesh Lall,et al.  Salprop: Salient object proposals via aggregated edge cues , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[2]  King Ngi Ngan,et al.  Improving object proposals with top-down cues , 2017, Signal Process. Image Commun..

[3]  Nader Karimi,et al.  Image retargeting using depth assisted saliency map , 2017, Signal Process. Image Commun..

[4]  Joseph J. Lim,et al.  Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[6]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[7]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[8]  Bernt Schiele,et al.  What Makes for Effective Detection Proposals? , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Sven Behnke,et al.  PyStruct: learning structured prediction in python , 2014, J. Mach. Learn. Res..

[10]  Xuelong Li,et al.  Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement , 2018, Pattern Recognit..

[11]  Ali Gooya,et al.  Supervised Saliency Map Driven Segmentation of Lesions in Dermoscopic Images , 2017, IEEE Journal of Biomedical and Health Informatics.

[12]  Vladlen Koltun,et al.  Geodesic Object Proposals , 2014, ECCV.

[13]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[14]  Junsong Yuan,et al.  Adobe Boxes: Locating Object Proposals Using Object Adobes , 2016, IEEE Transactions on Image Processing.

[15]  Yueting Zhuang,et al.  DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection , 2015, IEEE Transactions on Image Processing.

[16]  Matthew B. Blaschko,et al.  Learning a category independent object detection cascade , 2011, 2011 International Conference on Computer Vision.

[17]  Weisi Lin,et al.  Backward Registration-Based Aspect Ratio Similarity for Image Retargeting Quality Assessment , 2016, IEEE Transactions on Image Processing.

[18]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[19]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[20]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  Adrian Munteanu,et al.  Consistent video projection on curved displays , 2019, Signal Process. Image Commun..

[23]  Zhibo Chen,et al.  Full Reference Quality Assessment for Image Retargeting Based on Natural Scene Statistics Modeling and Bi-Directional Saliency Similarity , 2017, IEEE Transactions on Image Processing.

[24]  Shuyuan Yang,et al.  Object-level saliency detection with color attributes , 2016, Pattern Recognit..

[25]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, ACM Trans. Graph..

[26]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[27]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Charless C. Fowlkes,et al.  Oriented edge forests for boundary detection , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Tao Xiang,et al.  Making better use of edges via perceptual grouping , 2015, CVPR.

[31]  Min Li,et al.  Study of visual saliency detection via nonlocal anisotropic diffusion equation , 2015, Pattern Recognit..

[32]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[33]  Jitendra Malik,et al.  DeepBox: Learning Objectness with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Xing Cai,et al.  PDNet: Prior-Model Guided Depth-Enhanced Network for Salient Object Detection , 2018, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[35]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Esa Rahtu,et al.  Generating Object Segmentation Proposals Using Global and Local Search , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[39]  James M. Rehg,et al.  RIGOR: Reusing Inference in Graph Cuts for Generating Object Regions , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Santiago Manen,et al.  Prime Object Proposals with Randomized Prim's Algorithm , 2013, 2013 IEEE International Conference on Computer Vision.

[41]  C. Lawrence Zitnick,et al.  Fast Edge Detection Using Structured Forests , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[43]  Alexandros Iosifidis,et al.  Probabilistic saliency estimation , 2016, Pattern Recognit..

[44]  Olga Sorkine-Hornung,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH Asia '08.

[45]  Ming Yu,et al.  Image retargeting quality assessment based on content deformation measurement , 2018, Signal Process. Image Commun..

[46]  Guangming Shi,et al.  Nonlocal center-surround reconstruction-based bottom-up saliency estimation , 2013, 2013 IEEE International Conference on Image Processing.

[47]  Huimin Ma,et al.  Improving object proposals with multi-thresholding straddling expansion , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).