Saliency Detection via Topological Feature Modulated Deep Learning

The topological feature in an image, such as connectivity and adjacency, plays an important role in eye fixation detection. However, due to the scalar and additive nature of neurons to aggregate the node values in a local neighborhood, it is hard for convolutional neural networks (CNNs) to directly obtain and model the topological feature. Thus we adopt the topological feature which is pre-trained in accordance with the relationship between the figure and ground. Then we add a topological feature modulated convolutional layer into CNNs. By this way, the topological feature is automatically modulated by the deep features and well modeled by the CNNs. Experimental results show the proposed method surpasses the state-of-the-art by a big margin.

[1]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Naila Murray,et al.  End-to-End Saliency Mapping via Probability Distribution Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[4]  Aykut Erdem,et al.  Visual saliency estimation by nonlinearly integrating features using region covariances. , 2013, Journal of vision.

[5]  Chun Chen,et al.  What Is the Chance of Happening: A New Way to Predict Where People Look , 2010, ECCV.

[6]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[7]  Neil D. B. Bruce,et al.  A Deeper Look at Saliency: Feature Contrast, Semantics, and Beyond , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Guangming Shi,et al.  Bottom–Up Visual Saliency Estimation With Deep Autoencoder-Based Sparse Reconstruction , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Noel E. O'Connor,et al.  Shallow and Deep Convolutional Networks for Saliency Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Michael Dorr,et al.  Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Rong Li,et al.  Attention region detection based on closure prior in layered bit Planes , 2017, Neurocomputing.

[13]  R. Venkatesh Babu,et al.  DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations , 2015, IEEE Transactions on Image Processing.

[14]  Qi Zhao,et al.  SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  L Chen,et al.  Topological structure in visual perception. , 1982, Science.

[16]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Matthias Bethge,et al.  Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet , 2014, ICLR.

[18]  Stan Sclaroff,et al.  Saliency Detection: A Boolean Map Approach , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Shuo Wang,et al.  Predicting human gaze beyond pixels. , 2014, Journal of vision.

[20]  Nicolas Riche,et al.  Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.