Saliency prediction by Mahalanobis distance of topological feature on deep color components

Abstract A new saliency prediction method via extracting topological feature and calculating Mahalanobis distance on deep color components is presented in this paper. Specifically, four selectable schemes of color components are considered and a deep convolutional network is used to learn the best scheme. Then the topological feature maps of an input image are extracted on the learned color components by the analysis of connectivity and adjacency. To achieve the final saliency map, a new fusion method is proposed by calculating the Mahalanobis distance between the feature maps and their means with their covariance matrices rather than summating the feature maps linearly. The numerical and visual evaluation shows that a competitive performance compared with fourteen state-of-the-art models is achieved by the proposed method.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Yan Leng,et al.  Application of image correction and bit-plane fusion in generalized PCA based face recognition , 2007, Pattern Recognit. Lett..

[3]  Lin Li,et al.  Saliency Detection With Spaces of Background-Based Distribution , 2016, IEEE Signal Processing Letters.

[4]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  L Chen,et al.  Topological structure in visual perception. , 1982, Science.

[6]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Ali Borji,et al.  Exploiting local and global patch rarities for saliency detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[15]  Indriyati Atmosukarto,et al.  3D model retrieval with morphing-based geometric and topological feature maps , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[16]  Stan Sclaroff,et al.  Exploiting Surroundedness for Saliency Detection: A Boolean Map Approach , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Ling Huang,et al.  Saliency-based multi-feature modeling for semantic image retrieval , 2018, J. Vis. Commun. Image Represent..

[18]  R. Rosenholtz,et al.  The effect of background color on asymmetries in color search. , 2004, Journal of vision.

[19]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Sridha Sridharan,et al.  Deeper and wider fully convolutional network coupled with conditional random fields for scene labeling , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[21]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[22]  Yao Sun,et al.  Composing Semantic Collage for Image Retargeting , 2018, IEEE Transactions on Image Processing.

[23]  Christof Koch,et al.  Predicting human gaze using low-level saliency combined with face detection , 2007, NIPS.

[24]  Hanjiang Lai,et al.  Learning Adaptive Receptive Fields for Deep Image Parsing Network , 2017, CVPR.

[25]  Gert Kootstra,et al.  Paying Attention to Symmetry , 2008, BMVC.

[26]  Noel E. O'Connor,et al.  Shallow and Deep Convolutional Networks for Saliency Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Michael Dorr,et al.  Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Srinivas S. Kruthiventi,et al.  Saliency Unified: A Deep Architecture for simultaneous Eye Fixation Prediction and Salient Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Antón García-Díaz,et al.  Saliency from hierarchical adaptation through decorrelation and variance normalization , 2012, Image Vis. Comput..

[30]  Rong Li,et al.  Attention region detection based on closure prior in layered bit Planes , 2017, Neurocomputing.

[31]  Qi Zhao,et al.  Learning to predict eye fixations for semantic contents using multi-layer sparse network , 2014, Neurocomputing.

[32]  Bo Du,et al.  A Low-Rank and Sparse Matrix Decomposition-Based Mahalanobis Distance Method for Hyperspectral Anomaly Detection , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Neil D. B. Bruce,et al.  A Deeper Look at Saliency: Feature Contrast, Semantics, and Beyond , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[36]  Guangming Shi,et al.  Bottom–Up Visual Saliency Estimation With Deep Autoencoder-Based Sparse Reconstruction , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Ling Shao,et al.  Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement , 2015, IEEE Transactions on Image Processing.

[38]  Qi Zhao,et al.  SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Stan Sclaroff,et al.  Saliency Detection: A Boolean Map Approach , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[41]  Weisi Lin,et al.  A Saliency Detection Model Using Low-Level Features Based on Wavelet Transform , 2013, IEEE Transactions on Multimedia.

[42]  Nicolas Riche,et al.  Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[45]  Gang Wang,et al.  Deep Level Sets for Salient Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Ying Li,et al.  A weighted full-reference image quality assessment based on visual saliency , 2017, J. Vis. Commun. Image Represent..

[47]  Rainer Stiefelhagen,et al.  Quaternion-Based Spectral Saliency Detection for Eye Fixation Prediction , 2012, ECCV.