Multi-view adaptive semi-supervised feature selection with the self-paced learning

Abstract In recent decades, semi-supervised feature selection (SSFS) has gained extensive research in the field of machine learning and computer vision. Most SSFS algorithms are based on graph-based semi-supervised learning (GSSL), and their performance depend heavily on the quality of the Laplacian weight graph. However, the Laplacian weight graph can't be changed once it is constructed, which greatly restricts the performance of SSFS. To address this defect, in this paper we propose a novel Multi-view Adaptive Semi-supervised Feature Selection (MASFS) algorithm, which introduces the self-paced learning (SPL) into SSFS to make the Laplacian weight graph adaptively change according to the current predicted information. Meanwhile, the MASFS algorithm utilizes the multi-view learning to effectively explore the complementary and related information contained in different views to enhance SSFS performance. We propose a valid iterative algorithm for optimizing the objective function, followed by the convergence analysis and the complexity analysis. To illustrate the effectiveness of the MASFS algorithm, some experiments are carried out on NUS-WIDE dataset and MSRA-MM2.0 dataset and the experimental results indicate that MASFS has better performance than other SSFS algorithms based on GSSL.

[1]  Feiping Nie,et al.  Multi-View Correlated Feature Learning by Uncovering Shared Component , 2017, AAAI.

[2]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Chao Li,et al.  Active multi-kernel domain adaptation for hyperspectral image classification , 2017, Pattern Recognit..

[4]  Xiaoyi Jiang,et al.  GMDH-based semi-supervised feature selection for customer classification , 2017, Knowl. Based Syst..

[5]  Gavin C. Cawley,et al.  Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation , 2006, NIPS.

[6]  Qi Xie,et al.  Self-Paced Co-training , 2017, ICML.

[7]  Qiuqi Ruan,et al.  Sparse feature selection based on graph Laplacian for web image annotation , 2014, Image Vis. Comput..

[8]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[9]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[10]  Nannan Gu,et al.  Structure regularized self-paced learning for robust semi-supervised pattern classification , 2019, Neural Computing and Applications.

[11]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[13]  Bingbing Ni,et al.  Learning a Propagable Graph for Semisupervised Learning: Classification and Regression , 2012, IEEE Transactions on Knowledge and Data Engineering.

[14]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[15]  Wei Xu,et al.  Multi-modal self-paced learning for image classification , 2018, Neurocomputing.

[16]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[17]  Ran He,et al.  Nonnegative sparse coding for discriminative semi-supervised learning , 2011, CVPR 2011.

[18]  Lei Huang,et al.  Online semi-supervised annotation via proxy-based local consistency propagation , 2015, Neurocomputing.

[19]  Xianglong Liu,et al.  Multiple feature kernel hashing for large-scale visual search , 2014, Pattern Recognit..

[20]  Shuyuan Yang,et al.  Low-rank representation with local constraint for graph construction , 2013, Neurocomputing.

[21]  John D. Lafferty,et al.  Semi-supervised learning using randomized mincuts , 2004, ICML.

[22]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[23]  C. Grady,et al.  Event-related fMRI studies of episodic encoding and retrieval: Meta-analyses using activation likelihood estimation , 2009, Neuropsychologia.

[24]  Moritz A. Drupp,et al.  Indicator-Based Analysis of the Process Towards a University in Sustainable Development: A Case Study of the University of Tübingen (Germany) , 2015 .

[25]  Shih-Fu Chang,et al.  Graph construction and b-matching for semi-supervised learning , 2009, ICML '09.

[26]  Qiang Song,et al.  Modified Co-Training With Spectral and Spatial Views for Semisupervised Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Wai Keung Wong,et al.  Robust Latent Subspace Learning for Image Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Shiliang Sun,et al.  Robust Co-Training , 2011, Int. J. Pattern Recognit. Artif. Intell..

[29]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[30]  Yun Fu,et al.  Multi-view graph learning with adaptive label propagation , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[31]  Kun Zhan,et al.  Graph Learning for Multiview Clustering , 2018, IEEE Transactions on Cybernetics.

[32]  Congyan Lang,et al.  A Self-Paced Regularization Framework for Partial-Label Learning , 2016, IEEE Transactions on Cybernetics.

[33]  Daphne Koller,et al.  Self-Paced Learning for Latent Variable Models , 2010, NIPS.

[34]  Lei Huang,et al.  A general non-parametric active learning framework for classification on multiple manifolds , 2020, Pattern Recognit. Lett..

[35]  Deyu Meng,et al.  Semi-supervised learning through adaptive Laplacian graph trimming , 2017, Image Vis. Comput..

[36]  Nicu Sebe,et al.  Discriminating Joint Feature Analysis for Multimedia Data Understanding , 2012, IEEE Transactions on Multimedia.

[37]  Shichao Zhang,et al.  Feature selection by combining subspace learning with sparse representation , 2017, Multimedia Systems.

[38]  Meng Wang,et al.  MSRA-MM 2.0: A Large-Scale Web Multimedia Dataset , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[39]  Bo Wang,et al.  Deep Co-Training for Semi-Supervised Image Recognition , 2018, ECCV.

[40]  Lei Huang,et al.  Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search , 2016, IEEE Transactions on Image Processing.

[41]  Qiuqi Ruan,et al.  Semi-supervised sparse feature selection based on multi-view Laplacian regularization , 2015, Image Vis. Comput..

[42]  Feiping Nie,et al.  Semi-Supervised Feature Selection via Insensitive Sparse Regression with Application to Video Semantic Recognition , 2018, IEEE Transactions on Knowledge and Data Engineering.

[43]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.