Robust Discriminative multi-view K-means clustering with feature selection and group sparsity learning

With the rapid development of information technologies, more and more data are collected from multiple sources, which contain different perspectives of the data. To accurately explore the shared information among multiple views, K-means based multi-view clustering methods are designed and widely used in various applications for their simplicity and efficiency. However, all of these methods cluster data in the original high-dimensional feature space which is extremely time-consuming and sensitive to outliers, or cluster data in the embedded feature space for each view, which is hard to find the optimal reduced dimensionality. To solve these problems, we propose a robust discriminative multi-view K-means clustering with feature selection and group sparsity learning. Compared to the state-of-the-arts, the proposed algorithm has two advantages: 1) Discriminative K-means clustering and feature learning are integrated jointly into a single framework, where robust and accurate clustering results are obtained in the embedded feature space with an l2, 1-norm based loss function. 2) Group sparsity constraints are imposed to select the most relevant features and the most important views. We apply the proposed algorithm to serval kinds of multimedia understanding applications. Experimental results demonstrate the effectiveness of the proposed algorithm.

[1]  Rung-Ching Chen,et al.  Semi-supervised feature selection with exploiting shared information among multiple tasks , 2016, J. Vis. Commun. Image Represent..

[2]  Rung Ching Chen,et al.  Semi-supervised adaptive feature analysis and its application for multimedia understanding , 2018, Multimedia Tools and Applications.

[3]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[4]  Yi Yang,et al.  A Convex Formulation for Spectral Shrunk Clustering , 2015, AAAI.

[5]  Nicu Sebe,et al.  Web Image Annotation Via Subspace-Sparsity Collaborated Feature Selection , 2012, IEEE Transactions on Multimedia.

[6]  Lina Yao,et al.  Uncovering Locally Discriminative Structure for Feature Analysis , 2016, ECML/PKDD.

[7]  Xuelong Li,et al.  Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification , 2016, IJCAI.

[8]  Rongrong Ji,et al.  Nonnegative Spectral Clustering with Discriminative Regularization , 2011, AAAI.

[9]  Yueting Zhuang,et al.  Adaptive Unsupervised Multi-view Feature Selection for Visual Concept Recognition , 2012, ACCV.

[10]  Nicu Sebe,et al.  Knowledge Adaptation with PartiallyShared Features for Event DetectionUsing Few Exemplars , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yi Yang,et al.  Ranking with local regression and global alignment for cross media retrieval , 2009, ACM Multimedia.

[12]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[13]  Feiping Nie,et al.  Unsupervised Feature Selection via Unified Trace Ratio Formulation and K-means Clustering (TRACK) , 2014, ECML/PKDD.

[14]  Brendan J. Frey,et al.  Non-metric affinity propagation for unsupervised image categorization , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[16]  Shuyuan Yang,et al.  Global discriminative-based nonnegative spectral clustering , 2016, Pattern Recognit..

[17]  Feiping Nie,et al.  Re-Weighted Discriminatively Embedded $K$ -Means for Multi-View Clustering , 2017, IEEE Transactions on Image Processing.

[18]  Yang Yang,et al.  Robust (Semi) Nonnegative Graph Embedding , 2014, IEEE Transactions on Image Processing.

[19]  Xuelong Li,et al.  Unsupervised Feature Selection with Structured Graph Optimization , 2016, AAAI.

[20]  Xuan Li,et al.  Local and Global Discriminative Learning for Unsupervised Feature Selection , 2013, 2013 IEEE 13th International Conference on Data Mining.

[21]  Zi Huang,et al.  Robust Hashing With Local Models for Approximate Similarity Search , 2014, IEEE Transactions on Cybernetics.

[22]  Eun-Soo Kim,et al.  Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection , 2014, Multimedia Tools and Applications.

[23]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[24]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[25]  Feiping Nie,et al.  Discriminative Embedded Clustering: A Framework for Grouping High-Dimensional Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Shannon L. Risacher,et al.  Identifying quantitative trait loci via group-sparse multitask regression and feature selection: an imaging genetics study of the ADNI cohort , 2012, Bioinform..

[27]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[28]  Feiping Nie,et al.  Learning a subspace for clustering via pattern shrinking , 2013, Inf. Process. Manag..

[29]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[30]  Yi Yang,et al.  Multi-Class Active Learning by Uncertainty Sampling with Diversity Maximization , 2015, International Journal of Computer Vision.

[31]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[32]  Chenping Hou,et al.  Robust auto-weighted multi-view subspace clustering with common subspace representation matrix , 2017, PloS one.

[33]  Jian Zhang,et al.  Unsupervised spectral feature selection with l1-norm graph , 2016, Neurocomputing.

[34]  Nicu Sebe,et al.  Feature Selection for Multimedia Analysis by Sharing Information Among Multiple Tasks , 2013, IEEE Transactions on Multimedia.

[35]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[36]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[37]  Yi Yang,et al.  Image Classification by Cross-Media Active Learning With Privileged Information , 2016, IEEE Transactions on Multimedia.

[38]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[39]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[40]  Liang He,et al.  Semi-supervised minimum redundancy maximum relevance feature selection for audio classification , 2016, Multimedia Tools and Applications.

[41]  Yi Yang,et al.  Harmonizing Hierarchical Manifolds for Multimedia Document Semantics Understanding and Cross-Media Retrieval , 2008, IEEE Transactions on Multimedia.

[42]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[43]  Zi Huang,et al.  Multi-Feature Fusion via Hierarchical Regression for Multimedia Analysis , 2013, IEEE Transactions on Multimedia.

[44]  Rung Ching Chen,et al.  Semi-supervised multi-label feature selection via label correlation analysis with l1-norm graph embedding , 2017, Image Vis. Comput..

[45]  Feiping Nie,et al.  Discriminatively Embedded K-Means for Multi-view Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Feiping Nie,et al.  Orthogonal vs. uncorrelated least squares discriminant analysis for feature extraction , 2012, Pattern Recognit. Lett..

[47]  Shannon L. Risacher,et al.  Identifying disease sensitive and quantitative trait-relevant biomarkers from multidimensional heterogeneous imaging genetics data via sparse multimodal multitask learning , 2012, Bioinform..

[48]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.