Feature Map Quality Score Estimation Through Regression

Understanding the visual quality of a feature map plays a significant role in many active vision applications. Previous works mostly rely on object-level features, such as compactness, to estimate the quality score of a feature map. However, the compactness is leveraged on feature maps produced by salient object detection techniques where the maps tend to be compact. As a result, the compactness feature fails when the feature maps are blurry (e.g., fixation maps). In this paper, we regard the process of estimating the quality score of feature maps, specifically fixation maps, as a regression problem. After extracting several local, global, geometric, and positional characteristic features from a feature map, a model is learned using a random forest regressor to estimate the quality score of any unseen feature map. Our model is specifically tailored to estimate the quality of three types of maps: bottom-up, target, and contextual feature maps. These maps are produced for a large benchmark fixation data set of more than 900 challenging outdoor images. We demonstrate that our approach provides an accurate estimate of the quality of the abovementioned feature maps compared to the groundtruth data. In addition, we show that our proposed approach is useful in feature map integration for predicting human fixation. Instead of naively integrating all three feature maps when predicting human fixation, our proposed approach dynamically selects the best feature map with the highest estimated quality score on an individual image basis, thereby improving the fixation prediction accuracy.

[1]  Jian Sun,et al.  Face Alignment via Regressing Local Binary Features , 2016, IEEE Transactions on Image Processing.

[2]  Wei Zhang,et al.  The quest for the integration of visual saliency models in objective image quality assessment: A distraction power compensated combination strategy , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[3]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[5]  Simone Frintrop,et al.  Robust Object Detection at Regions of Interest with an Application in Ball Recognition , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[6]  S. Kastner,et al.  Interactions of Top-Down and Bottom-Up Mechanisms in Human Visual Cortex , 2011, The Journal of Neuroscience.

[7]  Gérard Biau,et al.  Analysis of a Random Forests Model , 2010, J. Mach. Learn. Res..

[8]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[9]  Larry S. Davis,et al.  Submodular Salient Region Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[11]  Ali Borji,et al.  Boosting bottom-up and top-down visual features for saliency estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Laurent Itti,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Rapid Biologically-inspired Scene Classification Using Features Shared with Visual Attention , 2022 .

[14]  Vibhav Vineet,et al.  Efficient Salient Region Detection with Soft Image Abstraction , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Hansang Lee,et al.  A novel method for salient object detection via compactness measurement , 2013, 2013 IEEE International Conference on Image Processing.

[16]  Allen Allport,et al.  Visual attention , 1989 .

[17]  Weisi Lin,et al.  Visual Saliency Detection With Free Energy Theory , 2015, IEEE Signal Processing Letters.

[18]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[19]  Hyunseung Choo,et al.  A critical review of selective attention: an interdisciplinary perspective , 2011, Artificial Intelligence Review.

[20]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Roseli A. Francelin Romero,et al.  Top-Down Biasing and Modulation for Object-Based Visual Attention , 2013, ICONIP.

[22]  Wei Xu,et al.  Look and Think Twice: Capturing Top-Down Visual Attention with Feedback Convolutional Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[24]  Laurent Itti,et al.  Saliency and Gist Features for Target Detection in Satellite Images , 2011, IEEE Transactions on Image Processing.

[25]  B. S. Manjunath,et al.  Learning top down scene context for visual attention modeling in natural images , 2013, 2013 IEEE International Conference on Image Processing.

[26]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[27]  Nuno Vasconcelos,et al.  Discriminant Saliency, the Detection of Suspicious Coincidences, and Applications to Visual Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Deepu Rajan,et al.  Salient Region Detection by Modeling Distributions of Color and Orientation , 2009, IEEE Transactions on Multimedia.

[29]  Xing Xie,et al.  Salient Region Detection Using Weighted Feature Maps Based on the Human Visual Attention Model , 2004, PCM.

[30]  Joanna Isabelle Olszewska,et al.  Multi-feature vector flow for active contour tracking , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[31]  Jeremy M Wolfe,et al.  Visual Attention , 2020, Computational Models for Cognitive Vision.

[32]  Huchuan Lu,et al.  Inner and Inter Label Propagation: Salient Object Detection in the Wild , 2015, IEEE Transactions on Image Processing.

[33]  John K. Tsotsos,et al.  A Computational Learning Theory of Active Object Recognition Under Uncertainty , 2012, International Journal of Computer Vision.

[34]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[35]  David Dagan Feng,et al.  Robust saliency detection via regularized random walks ranking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Xiaolin Hu,et al.  Feature Selection in Supervised Saliency Prediction , 2015, IEEE Transactions on Cybernetics.

[37]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[38]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[39]  Danica Kragic,et al.  An Active Vision System for Detecting, Fixating and Manipulating Objects in the Real World , 2010, Int. J. Robotics Res..

[40]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[42]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[43]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Krista A. Ehinger,et al.  Modelling search for people in 900 scenes: A combined source model of eye guidance , 2009 .

[45]  Mohammed Bennamoun,et al.  Linear Regression for Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[47]  Mengjie Zhang,et al.  Contextual-based top-down saliency feature weighting for target detection , 2016, Machine Vision and Applications.

[48]  Qi Wang,et al.  Tag-Saliency: Combining bottom-up and top-down information for saliency detection , 2014, Comput. Vis. Image Underst..