Adaptive Structural Co-regularization for Unsupervised Multi-view Feature Selection

With the advent of big data, there is an urgent need for methods and tools for integrative analyses of multi-modal or multi-view data. Of particular interest are unsupervised methods for parsimonious selection of non-redundant, complementary, and information-rich features from multi-view data. We introduce Adaptive Structural Co-Regularization Algorithm (ASCRA) for unsupervised multi-view feature selection. ASCRA jointly optimizes the embeddings of the different views so as to maximize their agreement with a consensus embedding which aims to simultaneously recover the latent cluster structure in the multi-view data while accounting for correlations between views. ASCRA uses the consensus embedding to guide efficient selection of features that preserve the latent cluster structure of the multi-view data. We establish ASCRA's convergence properties and analyze its computational complexity. The results of our experiments using several real-world and synthetic data sets suggest that ASCRA outperforms or is competitive with state-of-the-art unsupervised multi-view feature selection methods.

[1]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[2]  James M. Rehg,et al.  Where am I: Place instance and category recognition using spatial PACT , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Huan Liu,et al.  Unsupervised Feature Selection for Multi-View Data in Social Media , 2013, SDM.

[4]  Zhao Wang,et al.  Adaptive multi-view feature selection for human motion retrieval , 2016, Signal Process..

[5]  Huan Liu,et al.  Embedded Unsupervised Feature Selection , 2015, AAAI.

[6]  Huan Liu,et al.  Feature Selection , 2017, Encyclopedia of Machine Learning and Data Mining.

[7]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[8]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Xuelong Li,et al.  Multiview Clustering via Adaptively Weighted Procrustes , 2018, KDD.

[11]  Jieping Ye,et al.  Multi-Task Feature Learning Via Efficient l2, 1-Norm Minimization , 2009, UAI.

[12]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[13]  Kewei Cheng,et al.  Feature Selection , 2016, ACM Comput. Surv..

[14]  Feiping Nie,et al.  Multi-View Unsupervised Feature Selection with Adaptive Similarity and View Weight , 2017, IEEE Transactions on Knowledge and Data Engineering.

[15]  Huan Liu,et al.  Feature Selection for Clustering: A Review , 2018, Data Clustering: Algorithms and Applications.

[16]  Christoph Buchheim,et al.  Quadratic Outer Approximation for Convex Integer Programming with Box Constraints , 2013, SEA.

[17]  Kun He,et al.  Detecting Overlapping Communities from Local Spectral Subspaces , 2015, 2015 IEEE International Conference on Data Mining.

[18]  Vasant Honavar,et al.  Improving Image Captioning by Leveraging Knowledge Graphs , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[20]  Vasant Honavar,et al.  MEGAN: A Generative Adversarial Network for Multi-View Network Embedding , 2019, IJCAI.

[21]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[22]  Vasant Honavar,et al.  Min-redundancy and max-relevance multi-view feature selection for predicting ovarian cancer survival using multi-omics data , 2018, BMC Medical Genomics.

[23]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Vasant Honavar,et al.  Compositional Stochastic Average Gradient for Machine Learning and Related Applications , 2018, IDEAL.

[25]  Jingjing Li,et al.  Adaptive Collaborative Similarity Learning for Unsupervised Multi-view Feature Selection , 2018, IJCAI.

[26]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[27]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[28]  Chun-Hsiang Chuang,et al.  Minority Oversampling in Kernel Adaptive Subspaces for Class Imbalanced Datasets , 2018, IEEE Transactions on Knowledge and Data Engineering.

[29]  Shang-Lin Wu,et al.  Fuzzy Integral With Particle Swarm Optimization for a Motor-Imagery-Based Brain–Computer Interface , 2017, IEEE Transactions on Fuzzy Systems.

[30]  Nebojsa Jojic,et al.  LOCUS: learning object classes with unsupervised segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[31]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[32]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[33]  Vasant Honavar,et al.  Multi-view Network Embedding via Graph Factorization Clustering and Co-regularized Multi-view Agreement , 2018, 2018 IEEE International Conference on Data Mining Workshops (ICDMW).

[34]  Rafael Lazimy,et al.  Mixed-integer quadratic programming , 1982, Math. Program..

[35]  Xuelong Li,et al.  Feature selection with multi-view data: A survey , 2019, Inf. Fusion.

[36]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[37]  Yueting Zhuang,et al.  Adaptive Unsupervised Multi-view Feature Selection for Visual Concept Recognition , 2012, ACCV.

[38]  Mary Goldman,et al.  The UCSC cancer genomics browser: update 2011 , 2010, Nucleic Acids Res..

[39]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[40]  Feiping Nie,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Feature Selection via Joint Embedding Learning and Sparse Regression , 2022 .

[41]  Ming Shao,et al.  Robust Multi-view Representation: A Unified Perspective from Multi-view Learning to Domain Adaption , 2018, IJCAI.

[42]  Mengdi Wang,et al.  Finite-sum Composition Optimization via Variance Reduced Gradient Descent , 2016, AISTATS.

[43]  Shiliang Sun,et al.  Multi-view learning overview: Recent progress and new challenges , 2017, Inf. Fusion.

[44]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.