Multiview Subspace Clustering via Tensorial t-Product Representation

The ubiquitous information from multiple-view data, as well as the complementary information among different views, is usually beneficial for various tasks, for example, clustering, classification, denoising, and so on. Multiview subspace clustering is based on the fact that multiview data are generated from a latent subspace. To recover the underlying subspace structure, a successful approach adopted recently has been sparse and/or low-rank subspace clustering. Despite the fact that existing subspace clustering approaches may numerically handle multiview data, by exploring all possible pairwise correlation within views, high-order statistics that can only be captured by simultaneously utilizing all views are often overlooked. As a consequence, the clustering performance of the multiview data is compromised. To address this issue, in this paper, a novel multiview clustering method is proposed by using t-product in the third-order tensor space. First, we propose a novel tensor construction method to organize multiview tensorial data, to which the tensor-tensor product can be applied. Second, based on the circular convolution operation, multiview data can be effectively represented by a t-linear combination with sparse and low-rank penalty using “self-expressiveness.” Our extensive experimental results on face, object, digital image, and text data demonstrate that the proposed method outperforms the state-of-the-art methods for a range of criteria.

[1]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[2]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[3]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[4]  Derek Greene,et al.  A Matrix Factorization Approach for Integrating Multiple Data Views , 2009, ECML/PKDD.

[5]  Jia Xu,et al.  Spectral Clustering with a Convex Regularizer on Millions of Images , 2014, ECCV.

[6]  Martha White,et al.  Convex Multi-view Subspace Learning , 2012, NIPS.

[7]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[8]  M. Kilmer,et al.  Factorization strategies for third-order tensors , 2011 .

[9]  Yun Fu,et al.  Low-Rank Common Subspace for Multi-view Learning , 2014, 2014 IEEE International Conference on Data Mining.

[10]  Liang Wang,et al.  Multi-view clustering via pairwise sparse subspace representation , 2015, Neurocomputing.

[11]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[12]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[13]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[14]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[15]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Wei Peng,et al.  Clustering multi-way data via adaptive subspace iteration , 2008, CIKM '08.

[17]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[18]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[20]  Meng Wang,et al.  Topic driven multimodal similarity learning with multi-view voted convolutional features , 2018, Pattern Recognit..

[21]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[22]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[23]  Huan Liu,et al.  Subspace clustering for high dimensional data: a review , 2004, SKDD.

[24]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[25]  Weiwei Guo,et al.  Tensor Learning for Regression , 2012, IEEE Transactions on Image Processing.

[26]  Meng Wang,et al.  Visual Classification by ℓ1-Hypergraph Modeling , 2015, IEEE Trans. Knowl. Data Eng..

[27]  Junbin Gao,et al.  A Submodule Clustering Method for Multi-way Data by Sparse and Low-Rank Representation , 2016, ArXiv.

[28]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.

[29]  Jingjing Tang,et al.  Multiview Privileged Support Vector Machines , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Aristidis Likas,et al.  Kernel-Based Weighted Multi-view Clustering , 2012, 2012 IEEE 12th International Conference on Data Mining.

[31]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[33]  Misha Elena Kilmer,et al.  Novel Factorization Strategies for Higher Order Tensors: Implications for Compression and Recovery of Multi-linear Data , 2013, ArXiv.

[34]  L. Hubert,et al.  Comparing partitions , 1985 .

[35]  Yun Fu,et al.  Robust Multi-View Subspace Learning through Dual Low-Rank Decompositions , 2016, AAAI.

[36]  Shu Wu,et al.  Multiview Clustering via Unified and View-Specific Embeddings Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[38]  M. Kilmer,et al.  Tensor-Tensor Products with Invertible Linear Transforms , 2015 .

[39]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[40]  Misha Elena Kilmer,et al.  Clustering multi-way data: a novel algebraic approach , 2014, ArXiv.

[41]  Junbin Gao,et al.  Dual Graph Regularized Latent Low-Rank Representation for Subspace Clustering , 2015, IEEE Transactions on Image Processing.

[42]  Junbin Gao,et al.  Nonlinear low-rank representation on Stiefel manifolds , 2015 .

[43]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[46]  G. Arfken Mathematical Methods for Physicists , 1967 .

[47]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[48]  Shuicheng Yan,et al.  A Unified Alternating Direction Method of Multipliers by Majorization Minimization , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Bart De Moor,et al.  Multiview Partitioning via Tensor Methods , 2013, IEEE Transactions on Knowledge and Data Engineering.

[50]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[51]  Junbin Gao,et al.  Tensor LRR and Sparse Coding-Based Subspace Clustering , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[52]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[53]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[55]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[57]  Xiaochun Cao,et al.  Low-Rank Tensor Constrained Multiview Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[58]  Xiaoyan Sun,et al.  TenSR: Multi-dimensional Tensor Sparse Representation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Lin Wu,et al.  Robust Subspace Clustering for Multi-View Data by Exploiting Correlation Consensus , 2015, IEEE Transactions on Image Processing.

[60]  Nello Cristianini,et al.  Spectral Kernel Methods for Clustering , 2001, NIPS.