Tensor-Based Low-Dimensional Representation Learning for Multi-View Clustering

With the development of data collection techniques, multi-view clustering becomes an emerging research direction to improve the clustering performance. This paper has shown that leveraging multi-view information is able to provide a rich and comprehensive description. One of the core problems is how to sufficiently represent multi-view data in the analysis. In this paper, we introduce a tensor-based representation learning method for multi-view clustering (tRLMvC) that can unify heterogeneous and high-dimensional multi-view feature spaces to a low-dimensional shared latent feature space and improve multi-view clustering performance. To sufficiently capture plenty multi-view information, the tRLMvC represents multi-view data as a third-order tensor, expresses each tensorial data point as a sparse $t$ -linear combination of all data points with $t$ -product, and constructs a self-expressive tensor through reconstruction coefficients. The low-dimensional multi-view data representation in the shared latent feature space can be obtained via Tucker decomposition on the self-expressive tensor. These two parts are iteratively performed so that the interaction between self-expressive tensor learning and its factorization can be enhanced and the new representation can be effectively generated for clustering purpose. We conduct extensive experiments on eight multi-view data sets and compare the proposed model with the state-of-the-art methods. Experimental results have shown that tRLMvC outperforms the baselines in terms of various evaluation metrics.

[1]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[2]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[3]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[4]  William W. Hager,et al.  Updating the Inverse of a Matrix , 1989, SIAM Rev..

[5]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[6]  Stan Z. Li,et al.  Exclusivity-Consistency Regularized Multi-view Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jian Yu,et al.  Learning Transferred Weights From Co-Occurrence Data for Heterogeneous Transfer Learning , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Bart De Moor,et al.  Multiview Partitioning via Tensor Methods , 2013, IEEE Transactions on Knowledge and Data Engineering.

[9]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[10]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[11]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[12]  René Vidal,et al.  Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework , 2016, IEEE Transactions on Image Processing.

[13]  Shiliang Sun,et al.  Multi-view learning overview: Recent progress and new challenges , 2017, Inf. Fusion.

[14]  Misha Elena Kilmer,et al.  Clustering multi-way data: a novel algebraic approach , 2014, ArXiv.

[15]  Xuelong Li,et al.  Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification. , 2018, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[16]  Michael K. Ng,et al.  Inexact Alternating Direction Methods for Image Recovery , 2011, SIAM J. Sci. Comput..

[17]  Xiaochun Cao,et al.  Low-Rank Tensor Constrained Multiview Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Dean P. Foster,et al.  Multi-View Learning of Word Embeddings via CCA , 2011, NIPS.

[19]  M. Kilmer,et al.  Factorization strategies for third-order tensors , 2011 .

[20]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[21]  Yun Fu,et al.  Multi-View Clustering via Deep Matrix Factorization , 2017, AAAI.

[22]  Junbin Gao,et al.  Low-rank Multi-view Clustering in Third-Order Tensor Space , 2016, ArXiv.

[23]  Dongdai Lin,et al.  Robust Tensor Clustering with Non-Greedy Maximization , 2013, IJCAI.

[24]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[25]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[26]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[27]  Xiaochun Cao,et al.  Diversity-induced Multi-view Subspace Clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yuan Xie,et al.  On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization , 2016, International Journal of Computer Vision.

[29]  Xuelong Li,et al.  Multi-view Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[31]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[32]  Jeff A. Bilmes,et al.  On Deep Multi-View Representation Learning , 2015, ICML.

[33]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.