Learning feature fusion strategies for various image types to detect salient objects

Salient object detection is the task of automatically localizing objects of interests in a scene by suppressing the background information, which facilitates various machine vision applications such as object segmentation, recognition and tracking. Combining features from different feature-modalities has been demonstrated to enhance the performance of saliency prediction algorithms and different feature combinations are often suited to different types of images. However, existing saliency learning techniques attempt to apply a single feature combination across all image types and thus lose generalization in the test phase when considering unseen images. Learning classifier systems (LCSs) are an evolutionary machine learning technique that evolve a set of rules, based on a niched genetic reproduction, which collectively solve the problem. It is hypothesized that the LCS technique has the ability to autonomously learn different feature combinations for different image types. Hence, this paper further investigates the application of LCS for learning image dependent feature fusion strategies for the task of salient object detection. The obtained results show that the proposed method outperforms, through evolving generalized rules to compute saliency maps, the individual feature based methods and seven combinatorial techniques in detecting salient objects from three well known benchmark datasets of various types and difficulty levels. HighlightsIncorporated a novel input instance matching scheme in a learning classifier system.Effectively learned different feature combinations for various types of images.Outperformed nine individual feature based methods and seven combinatorial methods.The new method preserves more details of objects than the state-of-the-art methods.

[1]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Mengjie Zhang,et al.  XCSR with Computed Continuous Action , 2012, Australasian Conference on Artificial Intelligence.

[3]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[4]  Thomas Deselaers,et al.  What is an object? , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Tim Kovacs,et al.  Foundations of learning classifier systems: An introduction , 2005 .

[6]  R. K. Agrawal,et al.  A novel approach to combine features for salient object detection using constrained particle swarm optimization , 2014, Pattern Recognit..

[7]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2012, ECCV.

[9]  Ling Shao,et al.  Specific object retrieval based on salient regions , 2006, Pattern Recognit..

[10]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[11]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[12]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Will N. Browne,et al.  Salient object detection via spectral matting , 2016, Pattern Recognit..

[14]  Jun Yu,et al.  Pairwise constraints based multiview features fusion for scene classification , 2013, Pattern Recognit..

[15]  Mengjie Zhang,et al.  Learning complex, overlapping and niche imbalance Boolean problems using XCS-based classifier systems , 2013, Evol. Intell..

[16]  E GoldbergDavid,et al.  Generalization in the XCSF Classifier System , 2007 .

[17]  Christof Koch,et al.  Learning visual saliency by combining feature maps in a nonlinear manner using AdaBoost. , 2012, Journal of vision.

[18]  Tao Mei,et al.  Learning salient visual word for scalable mobile image retrieval , 2015, Pattern Recognit..

[19]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[20]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Mengjie Zhang,et al.  Salient object detection using learning classifiersystems that compute action mappings , 2014, GECCO.

[22]  Will N. Browne,et al.  Extending XCS with Cyclic Graphs for Scalability on Complex Boolean Problems , 2017, Evolutionary Computation.

[23]  Masahiro Takei,et al.  Human resource development and visualization , 2009, J. Vis..

[24]  Huchuan Lu,et al.  Bayesian Saliency via Low and mid Level Cues , 2022 .

[25]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[26]  Christof Koch,et al.  Learning a saliency map using fixated locations in natural scenes. , 2011, Journal of vision.

[27]  Lihi Zelnik-Manor,et al.  Puzzle‐like Collage , 2010, Comput. Graph. Forum.

[28]  Vibhav Vineet,et al.  Efficient Salient Region Detection with Soft Image Abstraction , 2013, 2013 IEEE International Conference on Computer Vision.

[29]  Ali Borji,et al.  Boosting bottom-up and top-down visual features for saliency estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[31]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[33]  Simone Frintrop,et al.  Center-surround divergence of feature statistics for salient object detection , 2011, 2011 International Conference on Computer Vision.

[34]  Marco Colombetti,et al.  What Is a Learning Classifier System? , 1999, Learning Classifier Systems.

[35]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[36]  Will N. Browne,et al.  Optimizing visual attention models for predicting human fixations using Genetic Algorithms , 2013, 2013 IEEE Congress on Evolutionary Computation.

[37]  Will N. Browne,et al.  Combining object-based local and global feature statistics for salient object search , 2013, 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013).

[38]  Xiao Bai,et al.  Discriminative Features for Image Classification and Retrieval , 2011, 2011 Sixth International Conference on Image and Graphics.

[39]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[40]  Ruth Kimchi,et al.  Figure-Ground Segmentation Can Occur Without Attention , 2008, Psychological science.

[41]  Feng Liu,et al.  Comparing Salient Object Detection Results without Ground Truth , 2014, ECCV.

[42]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[44]  Luigi Barone,et al.  On XCSR for electronic fraud detection , 2012, Evol. Intell..

[45]  Nanning Zheng,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[46]  David Page,et al.  Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals , 2013, ECML/PKDD.

[47]  Daniele Loiacono,et al.  Classifier systems that compute action mappings , 2007, GECCO '07.

[48]  Jun Zhou,et al.  Object Detection Via Structural Feature Selection and Shape Model , 2013, IEEE Transactions on Image Processing.

[49]  Rama Chellappa,et al.  Salient views and view-dependent dictionaries for object recognition , 2015, Pattern Recognit..

[50]  Mengjie Zhang,et al.  Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems , 2014, IEEE Transactions on Evolutionary Computation.

[51]  Hussein A. Abbass,et al.  Intrusion detection with evolutionary learning classifier systems , 2009, Natural Computing.

[52]  Xiao Bai,et al.  In Search of Perceptually Salient Groupings , 2011, IEEE Transactions on Image Processing.

[53]  Larry Bull,et al.  Applications of Learning Classifier Systems , 2004 .

[54]  Mengjie Zhang,et al.  Transparent, Online Image Pattern Classification Using a Learning Classifier System , 2011, EvoApplications.

[55]  Daniele Loiacono,et al.  Generalization in the XCSF Classifier System: Analysis, Improvement, and Extension , 2007, Evolutionary Computation.

[56]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Huchuan Lu,et al.  Salient object detection via global and local cues , 2015, Pattern Recognit..

[59]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[60]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.