A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous Counterparts

For a given image, it is a challenging task to generate its corresponding counterpart with visual inconspicuous modification. The complexity of this problem reasons from the high correlativity between the editing operations and vision perception. Essentially, a significant requirement that should be emphasized is how to make the object modifications hard to be found visually in the generative counterparts. In this article, we propose a novel dual-domain perceptual framework to generate visual inconspicuous counterparts, which applies the perceptual bidirectional similarity metric (PBSM) and appearance similarity metric (ASM) to create the dual-domain perception error minimization model. The candidate targets are yielded by the well-known PatchMatch model with the strokes-based interactions and selective object library. By the dual-perceptual evaluation index, all candidate targets are sorted to select out the best result. For demonstration, a series of objective and subjective measurements are used to evaluate the performance of our framework.

[1]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[2]  Olga Sorkine-Hornung,et al.  Visual media retargeting , 2009, SIGGRAPH ASIA Courses.

[3]  Jian-Jiun Ding,et al.  Salient Region Detection Improved by Principle Component Analysis and Boundary Information , 2013, IEEE Transactions on Image Processing.

[4]  Svetlana Lazebnik,et al.  Superparsing , 2010, International Journal of Computer Vision.

[5]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[6]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[7]  Song-Chun Zhu,et al.  Abstract painting with interactive control of perceptual entropy , 2013, TAP.

[8]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[9]  Huchuan Lu,et al.  Bayesian Saliency via Low and mid Level Cues , 2022 .

[10]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[13]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[14]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[15]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[16]  Shipeng Li,et al.  A bag-of-objects retrieval model for web image search , 2012, ACM Multimedia.

[17]  Marc Alexa,et al.  PhotoSketch: a sketch based image query and compositing system , 2009, SIGGRAPH '09.

[18]  Shi-Min Hu,et al.  ImageAdmixture: Putting Together Dissimilar Objects from Groups , 2012, IEEE Transactions on Visualization and Computer Graphics.

[19]  Li Liu,et al.  Corruptive Artifacts Suppression for Example-Based Color Transfer , 2014, IEEE Transactions on Multimedia.

[20]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[21]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, ACM Trans. Graph..

[22]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Mubarak Shah,et al.  A holistic approach to aesthetic enhancement of photographs , 2011, TOMCCAP.

[24]  Olga Sorkine-Hornung,et al.  A comparative study of image retargeting , 2010, ACM Trans. Graph..

[25]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[26]  Sylvain Paris,et al.  Error-Tolerant Image Compositing , 2010, International Journal of Computer Vision.

[27]  Xuelong Li,et al.  Color to Gray: Visual Cue Preservation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[29]  Zeev Farbman,et al.  Coordinates for instant image cloning , 2009, ACM Trans. Graph..

[30]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[32]  William T. Freeman,et al.  The patch transform and its applications to image editing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[34]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[35]  Dani Lischinski,et al.  "Mind the gap" , 2013, ACM Trans. Graph..

[36]  Li Xu,et al.  Mutual-Structure for Joint Filtering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Song-Chun Zhu,et al.  From image parsing to painterly rendering , 2009, TOGS.

[38]  Qi Tian,et al.  Scalable Object Retrieval with Compact Image Representation from Generic Object Regions , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[39]  Ralph R. Martin,et al.  PatchNet: a patch-based image representation for interactive library-driven image editing , 2013, ACM Trans. Graph..

[40]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[41]  Shi-Min Hu,et al.  Data‐Driven Object Manipulation in Images , 2012, Comput. Graph. Forum.

[42]  Cewu Lu,et al.  Contrast Preserving Decolorization with Perception-Based Quality Metrics , 2014, International Journal of Computer Vision.

[43]  Li Xu,et al.  A sparse control model for image and video editing , 2013, ACM Trans. Graph..

[44]  Adam Finkelstein,et al.  The PatchMatch randomized matching algorithm for image manipulation , 2011, Commun. ACM.

[45]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Shi-Min Hu,et al.  Aesthetic Image Enhancement by Dependence-Aware Object Recomposition , 2013, IEEE Transactions on Multimedia.

[47]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.