Salient Region Guided Blind Image Sharpness Assessment

Salient regions provide important cues for scene understanding to the human vision system. However, whether the detected salient regions are helpful in image blur estimation is unknown. In this study, a salient region guided blind image sharpness assessment (BISA) framework is proposed, and the effect of the detected salient regions on the BISA performance is investigated. Specifically, three salient region detection (SRD) methods and ten BISA models are jointly explored, during which the output saliency maps from SRD methods are re-organized as the input of BISA models. Consequently, the change in BISA metric values can be quantified and then directly related to the difference in BISA model inputs. Finally, experiments are conducted on three Gaussian blurring image databases, and the BISA prediction performance is evaluated. The comparison results indicate that salient region input can help achieve a close and sometimes superior performance to a BISA model over the whole image input. When using the center region input as the baseline, the detected salient regions from the saliency optimization from robust background detection (SORBD) method lead to consistently better score prediction, regardless of the BISA model. Based on the proposed hybrid framework, this study reveals that saliency detection benefits image blur estimation, while how to properly incorporate SRD methods and BISA models to improve the score prediction will be explored in our future work.

[1]  Xiaohui Li,et al.  Multi-Scale Global Contrast CNN for Salient Object Detection , 2020, Sensors.

[2]  D. Chandler Seven Challenges in Image Quality Assessment: Past, Present, and Future Research , 2013 .

[3]  Alex ChiChung Kot,et al.  A Fast Approach for No-Reference Image Sharpness Assessment Based on Maximum Local Variation , 2014, IEEE Signal Processing Letters.

[4]  Yang Liu,et al.  Depth-aware salient object detection using anisotropic center-surround difference , 2015, Signal Process. Image Commun..

[5]  Phong V. Vu,et al.  A Fast Wavelet-Based Algorithm for Global and Local Image Sharpness Estimation , 2012, IEEE Signal Processing Letters.

[6]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[7]  Xiaochun Cao,et al.  Depth Enhanced Saliency Detection Method , 2014, ICIMCS '14.

[8]  J Driver,et al.  A selective review of selective attention research from the past century. , 2001, British journal of psychology.

[9]  King Ngi Ngan,et al.  A Co-Saliency Model of Image Pairs , 2011, IEEE Transactions on Image Processing.

[10]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Weisi Lin,et al.  No-Reference Image Blur Assessment Based on Discrete Orthogonal Moments , 2016, IEEE Transactions on Cybernetics.

[12]  Minglun Gong,et al.  No-reference image sharpness assessment based on discrepancy measures of structural degradation , 2020, J. Vis. Commun. Image Represent..

[13]  Simone Frintrop,et al.  Center-surround divergence of feature statistics for salient object detection , 2011, 2011 International Conference on Computer Vision.

[14]  Laurent Itti,et al.  Superior colliculus encodes visual saliency before the primary visual cortex , 2017, Proceedings of the National Academy of Sciences.

[15]  Lina J. Karam,et al.  A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection (CPBD) , 2011, IEEE Transactions on Image Processing.

[16]  Patrick Le Callet,et al.  A coherent computational approach to model bottom-up visual attention , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[18]  Zhuowen Tu,et al.  Unsupervised object class discovery via saliency-guided multiple class learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Dewen Hu,et al.  Salient Region Detection Using Diffusion Process on a Two-Layer Sparse Graph , 2017, IEEE Transactions on Image Processing.

[20]  Michael Ying Yang,et al.  Exploiting global priors for RGB-D saliency detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Changyu Duan,et al.  A pooling-based feature pyramid network for salient object detection , 2021, Image Vis. Comput..

[22]  Yaoqin Xie,et al.  CNN-GRNN for Image Sharpness Assessment , 2016, ACCV Workshops.

[23]  Radomír Mech,et al.  Unconstrained Salient Object Detection via Proposal Subset Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[25]  Tsung-Jung Liu,et al.  Study of Visual Quality Assessment on Pattern Images: Subjective Evaluation and Visual Saliency Effects , 2018, IEEE Access.

[26]  Gang Wang,et al.  Deep Level Sets for Salient Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Zhaoping Li,et al.  Neural Activities in V1 Create a Bottom-Up Saliency Map , 2012, Neuron.

[28]  Damon M. Chandler,et al.  ${\bf S}_{3}$: A Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images , 2012, IEEE Transactions on Image Processing.

[29]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[30]  Eui-Nam Huh,et al.  Center-Emphasized Visual Saliency and a Contrast-Based Full Reference Image Quality Index , 2018, Symmetry.

[31]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[32]  Weisi Lin,et al.  Image Sharpness Assessment by Sparse Representation , 2016, IEEE Transactions on Multimedia.

[33]  Gong Cheng,et al.  Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[34]  Wei Zhang,et al.  The Application of Visual Saliency Models in Objective Image Quality Assessment: A Statistical Evaluation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Yizhou Yu,et al.  Visual Saliency Detection Based on Multiscale Deep CNN Features , 2016, IEEE Transactions on Image Processing.

[36]  Kwang Nam Choi,et al.  SRIS: Saliency-Based Region Detection and Image Segmentation of COVID-19 Infected Cases , 2020, IEEE Access.

[37]  LinLin Shen,et al.  Visual-Patch-Attention-Aware Saliency Detection , 2015, IEEE Transactions on Cybernetics.

[38]  Weisi Lin,et al.  Which Has Better Visual Quality: The Clear Blue Sky or a Blurry Animal? , 2019, IEEE Transactions on Multimedia.

[39]  Junyu Dong,et al.  Image retrieval using wavelet-based salient regions , 2011 .

[40]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[41]  Lei Wang,et al.  A shallow convolutional neural network for blind image sharpness assessment , 2017, PloS one.

[42]  Weisi Lin,et al.  No-Reference and Robust Image Sharpness Evaluation Based on Multiscale Spatial and Spectral Features , 2017, IEEE Transactions on Multimedia.

[43]  Weisi Lin,et al.  No-Reference Image Sharpness Assessment in Autoregressive Parameter Space , 2015, IEEE Transactions on Image Processing.

[44]  Fuzheng Yang,et al.  Linking visual saliency deviation to image quality degradation: A saliency deviation-based image quality index , 2019, Signal Process. Image Commun..

[45]  Stefan Winkler,et al.  Perceptual blur and ringing metrics: application to JPEG2000 , 2004, Signal Process. Image Commun..

[46]  Jiajia Wu,et al.  Saliency Detection with Bilateral Absorbing Markov Chain Guided by Depth Information , 2021, Sensors.

[47]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Yaoqin Xie,et al.  Evaluation of no-reference models to assess image sharpness , 2017, 2017 IEEE International Conference on Information and Automation (ICIA).

[49]  Ming-Hsuan Yang,et al.  PiCANet: Pixel-Wise Contextual Attention Learning for Accurate Saliency Detection , 2018, IEEE Transactions on Image Processing.

[50]  Zhou Wang,et al.  Image Sharpness Assessment Based on Local Phase Coherence , 2013, IEEE Transactions on Image Processing.

[51]  Zhaoping Li A saliency map in primary visual cortex , 2002, Trends in Cognitive Sciences.

[52]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.

[53]  Konstantinos N Plataniotis,et al.  Encoding Visual Sensitivity by MaxPol Convolution Filters for Image Sharpness Assessment , 2018, IEEE Transactions on Image Processing.

[54]  Nuno Vasconcelos,et al.  Learning Optimal Seeds for Diffusion-Based Salient Object Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Chao Gao,et al.  BASNet: Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Feng Liu,et al.  No-reference Image Blur Assessment Based on Multi-scale Spatial Local Features , 2020, KSII Trans. Internet Inf. Syst..

[57]  Xiaojun Wu,et al.  No-reference image blur index based on singular value curve , 2014, J. Vis. Commun. Image Represent..

[58]  Chun Qi,et al.  Saliency detection based on global and local short-term sparse representation , 2016, Neurocomputing.

[59]  Huchuan Lu,et al.  Graph-Regularized Saliency Detection With Convex-Hull-Based Center Prior , 2013, IEEE Signal Processing Letters.

[60]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[61]  Hans-Jürgen Zepernick,et al.  Framework for optimal region of interest-based quality assessment in wireless imaging , 2010, J. Electronic Imaging.

[62]  Henrik I. Christensen,et al.  Computational visual attention systems and their cognitive foundations: A survey , 2010, TAP.

[63]  Alan C. Bovik,et al.  Visual Importance Pooling for Image Quality Assessment , 2009, IEEE Journal of Selected Topics in Signal Processing.

[64]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[65]  Yaoqin Xie,et al.  Transferring deep neural networks for the differentiation of mammographic breast lesions , 2018, Science China Technological Sciences.