Focus-and-Context Skeleton-based Image Simplification using Saliency Maps

Medial descriptors offer a promising way for representing, simplifying, manipulating, and compressing images. However, to date, these have been applied in a global manner that is oblivious to salient features. In this paper, we adapt medial descriptors to use the information provided by saliency maps to selectively simplify and encode an image while preserving its salient regions. This allows us to improve the trade-off between compression ratio and image quality as compared to the standard dense-skeleton method while keeping perceptually salient features, in a focus-and-context manner. We show how our method can be combined with JPEG to increase overall compression rates at the cost of a slightly lower image quality. We demonstrate our method on a benchmark composed of a broad set of images.

[1]  David Zhang,et al.  A comprehensive evaluation of full reference image quality assessment algorithms , 2012, 2012 19th IEEE International Conference on Image Processing.

[2]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[3]  Alexandru Telea,et al.  Quantitative Evaluation of Dense Skeletons for Image Compression , 2020, Inf..

[4]  Patrick Le Callet,et al.  Visual Attention and Applications in Multimedia Technologies , 2013, Proceedings of the IEEE.

[5]  Alexandru Telea,et al.  A Dense Medial Descriptor for Image Analysis , 2013, VISAPP.

[6]  Yael Pritch,et al.  Content-aware compression using saliency-driven image retargeting , 2013, 2013 IEEE International Conference on Image Processing.

[7]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[8]  Ingrid Heynderickx,et al.  How Does Image Content Affect the Added Value of Visual Attention in Objective Image Quality Assessment? , 2013, IEEE Signal Processing Letters.

[9]  Jorge Stolfi,et al.  The image foresting transform: theory, algorithms, and applications , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Holger Voos,et al.  When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking , 2020, Sensors.

[11]  Michal Podpora,et al.  YUV vs. RGB - A Comparison Of Lossy Compressions For Human-Oriented Man-Machine Interfaces , 2009 .

[12]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[13]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[14]  H. Nobuhara,et al.  Color image compression/reconstruction by YUV fuzzy wavelets , 2004, IEEE Annual Meeting of the Fuzzy Information, 2004. Processing NAFIPS '04..

[15]  Li Xu,et al.  Hierarchical Image Saliency Detection on Extended CSSD , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[17]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  James H. Elder,et al.  Design and perceptual validation of performance measures for salient object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[19]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2016, IEEE transactions on pattern analysis and machine intelligence.

[20]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2017, IEEE transactions on pattern analysis and machine intelligence.

[22]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Alireza Alaei,et al.  Image quality assessment based on regions of interest , 2017, Signal Image Video Process..

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  ITSELF: Iterative Saliency Estimation fLexible Framework , 2020, ArXiv.

[26]  Patrick Le Callet,et al.  Perceived interest and overt visual attention in natural images , 2015, Signal Process. Image Commun..

[27]  Chaofeng Li,et al.  Content-weighted video quality assessment using a three-component image model , 2010, J. Electronic Imaging.

[28]  Detlev Marpe,et al.  Performance analysis of HEVC-based intra coding for still image compression , 2012, 2012 Picture Coding Symposium.

[29]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[30]  Alexandru Telea,et al.  Effcient and Effective Automated Digital Hair Removal from Dermoscopy Images , 2016 .

[31]  Krista A. Ehinger,et al.  Hypergraph Optimization for Salient Region Detection Based on Foreground and Background Queries , 2018, IEEE Access.

[32]  Alexandru Telea Feature Preserving Smoothing of Shapes Using Saliency Skeletons , 2012, Visualization in Medicine and Life Sciences II.

[33]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[34]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Luciano da Fontoura Costa,et al.  Shape Analysis and Classification: Theory and Practice , 2000 .

[36]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[37]  A. Diana Andrushia,et al.  Saliency-Based Image Compression Using Walsh–Hadamard Transform (WHT) , 2018 .

[38]  Alexandru Telea,et al.  An Augmented Fast Marching Method for Computing Skeletons and Centerlines , 2002, VisSym.

[39]  Aleksandra Kawala-Janik,et al.  YUV vs RGB-Choosing a Color Space for Human-Machine Interaction , 2014, FedCSIS.

[40]  David Minnen,et al.  Variable Rate Image Compression with Recurrent Neural Networks , 2015, ICLR.

[41]  Alexandru Telea,et al.  Automated Digital Hair Removal by Threshold Decomposition and Morphological Analysis , 2015, ISMM.

[42]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.