COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction

In this paper, we study two challenging problems in incomplete multi-view clustering analysis, namely, i) how to learn an informative and consistent representation among different views without the help of labels and ii) how to recover the missing views from data. To this end, we propose a novel objective that incorporates representation learning and data recovery into a unified framework from the view of information theory. To be specific, the informative and consistent representation is learned by maximizing the mutual information across different views through contrastive learning, and the missing views are recovered by minimizing the conditional entropy of different views through dual prediction. To the best of our knowledge, this could be the first work to provide a theoretical framework that unifies the consistent representation learning and cross-view data recovery. Extensive experimental results show the proposed method remarkably outperforms 10 competitive multi-view clustering methods on four challenging datasets. The code is available at https://pengxi.me.

[1]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  Songcan Chen,et al.  Doubly Aligned Incomplete Multi-view Clustering , 2018, IJCAI.

[3]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[4]  Zenglin Xu,et al.  Partition level multiview subspace clustering , 2019, Neural Networks.

[5]  Xu Ji,et al.  Invariant Information Clustering for Unsupervised Image Classification and Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Cai Xu,et al.  Adversarial Incomplete Multi-view Clustering , 2019, IJCAI.

[7]  Chang Tang,et al.  Efficient and Effective Regularized Incomplete Multi-View Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xiaochun Cao,et al.  DM2C: Deep Mixed-Modal Clustering , 2019, NeurIPS.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Feiping Nie,et al.  Detecting Coherent Groups in Crowd Scenes by Multiview Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Dezhong Peng,et al.  Multi-View Linear Discriminant Analysis Network , 2019, IEEE Transactions on Image Processing.

[13]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[15]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[16]  Xinbo Gao,et al.  Triplet-Based Deep Hashing Network for Cross-Modal Retrieval , 2018, IEEE Transactions on Image Processing.

[17]  Hao Wang,et al.  Spectral Perturbation Meets Incomplete Multi-view Data , 2019, IJCAI.

[18]  Sham M. Kakade,et al.  An Information Theoretic Framework for Multi-view Learning , 2008, COLT.

[19]  Wei Liu,et al.  Multi-View Matrix Decomposition: A New Scheme for Exploring Discriminative Information , 2015, IJCAI.

[20]  Ruslan Salakhutdinov,et al.  Self-supervised Learning from a Multi-view Perspective , 2020, ICLR.

[21]  Huazhu Fu,et al.  Reciprocal Multi-Layer Subspace Learning for Multi-View Clustering , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Dacheng Tao,et al.  Multi-view Self-Paced Learning for Clustering , 2015, IJCAI.

[23]  Wei-Yun Yau,et al.  Structured AutoEncoders for Subspace Clustering , 2018, IEEE Transactions on Image Processing.

[24]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[25]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[26]  Laurens van der Maaten,et al.  Self-Supervised Learning of Pretext-Invariant Representations , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Jiancheng Lv,et al.  COMIC: Multi-view Clustering Without Parameter Selection , 2019, ICML.

[28]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[31]  Jeff A. Bilmes,et al.  On Deep Multi-View Representation Learning , 2015, ICML.

[32]  Yun Fu,et al.  Partial Multi-view Clustering via Consistent GAN , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[33]  Hong Liu,et al.  Unified Embedding Alignment with Missing Views Inferring for Incomplete Multi-View Clustering , 2019, AAAI.

[34]  Jiashi Feng,et al.  Deep Clustering With Sample-Assignment Invariance Prior , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Xi Peng,et al.  Partially View-aligned Representation Learning with Noise-robust Contrastive Loss , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Weiwei Liu,et al.  Sparse Embedded k-Means Clustering , 2017, NIPS.

[37]  Zheng Zhang,et al.  CDIMC-net: Cognitive Deep Incomplete Multi-view Clustering Network , 2020, IJCAI.

[38]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[39]  Shao-Yuan Li,et al.  Partial Multi-View Clustering , 2014, AAAI.

[40]  Dezhong Peng,et al.  Joint Versus Independent Multiview Hashing for Cross-View Retrieval , 2020, IEEE Transactions on Cybernetics.

[41]  Philip S. Yu,et al.  Multiple Incomplete Views Clustering via Weighted Nonnegative Matrix Factorization with L2, 1 Regularization , 2015, ECML/PKDD.

[42]  Joey Tianyi Zhou,et al.  Partially View-aligned Clustering , 2020, NeurIPS.

[43]  Yi-Dong Shen,et al.  End-to-End Adversarial-Attention Network for Multi-Modal Clustering , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Ming Shao,et al.  Cross-View Projective Dictionary Learning for Person Re-Identification , 2015, IJCAI.

[45]  Jie Zhang,et al.  Online Low-Rank Representation Learning for Joint Multi-Subspace Recovery and Clustering , 2018, IEEE Transactions on Image Processing.

[46]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[47]  Shaogang Gong,et al.  Deep Semantic Clustering by Partition Confidence Maximisation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[49]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[50]  Huazhu Fu,et al.  AE2-Nets: Autoencoder in Autoencoder Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Yun Fu,et al.  Incomplete Multi-Modal Visual Data Grouping , 2016, IJCAI.

[52]  Dacheng Tao,et al.  Multi-View Learning With Incomplete Views , 2015, IEEE Transactions on Image Processing.

[53]  Ling Shao,et al.  Binary Multi-View Clustering , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[55]  Dezhong Peng,et al.  Contrastive Clustering , 2021, AAAI.