ANIMC: A Soft Approach for Autoweighted Noisy and Incomplete Multiview Clustering

Multiview clustering has wide real-world applications because it can process data from multiple sources. However, these data often contain missing instances and noises, which are ignored by most multiview clustering methods. Missing instances may make these methods difficult to use directly, and noises will lead to unreliable clustering results. In this article, we propose a novel autoweighted noisy and incomplete multiview clustering (ANIMC) approach via a <italic>soft</italic> autoweighted strategy and a doubly <italic>soft</italic> regular regression model. First, by designing adaptive semiregularized nonnegative matrix factorization, the soft autoweighted strategy assigns a <italic>proper</italic> weight to each view and adds a soft boundary to balance the influence of noises and incompleteness. Second, by proposing <inline-formula><tex-math notation="LaTeX">$\theta$</tex-math></inline-formula>-norm, the doubly soft regularized regression model adjusts the sparsity of our model by choosing different <inline-formula><tex-math notation="LaTeX">$\theta$</tex-math></inline-formula>. Compared with previous methods, ANIMC has three unique advantages: 1) it is a soft algorithm to adjust our approach in different scenarios, thereby improving its generalization ability; 2) it automatically learns a proper weight for each view, thereby reducing the influence of noises; 3) it performs doubly soft regularized regression that aligns the same instances in different views, thereby decreasing the impact of missing instances. Extensive experimental results show its superior advantages over other state-of-the-art works.

[1]  Wenbo Xu,et al.  Structured Graph Learning for Scalable Subspace Clustering: From Single View to Multiview , 2021, IEEE Transactions on Cybernetics.

[2]  Dapeng Oliver Wu,et al.  Unbalanced Incomplete Multi-View Clustering Via the Scheme of View Evolution: Weak Views are Meat; Strong Views Do Eat , 2020, IEEE Transactions on Emerging Topics in Computational Intelligence.

[3]  Simon Carbonnelle,et al.  Intraclass clustering: an implicit learning ability that regularizes DNNs , 2021, ICLR.

[4]  Bob Zhang,et al.  Adaptive Graph Completion Based Incomplete Multi-View Clustering , 2020, IEEE Transactions on Multimedia.

[5]  Chang Tang,et al.  Efficient and Effective Regularized Incomplete Multi-View Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Zhenyu He,et al.  Semi-Supervised Multi-View Deep Discriminant Representation Learning , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Yue Gao,et al.  Deep Multi-View Enhancement Hashing for Image Retrieval , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xiangliang Zhang,et al.  Individuality- and Commonality-Based Multiview Multilabel Learning , 2019, IEEE Transactions on Cybernetics.

[9]  Shi-Jinn Horng,et al.  Tri-regularized nonnegative matrix tri-factorization for co-clustering , 2021, Knowl. Based Syst..

[10]  Dapeng Oliver Wu,et al.  V\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$^3$\end{document}H: View Variation and View Heredity for Incomplet , 2020, IEEE Transactions on Artificial Intelligence.

[11]  Dacheng Tao,et al.  Self-Supervised Pose Adaptation for Cross-Domain Image Animation , 2020, IEEE Transactions on Artificial Intelligence.

[12]  Rong Wang,et al.  Auto-weighted multi-view clustering via spectral embedding , 2020, Neurocomputing.

[13]  Yue Zhao,et al.  Intra- and Inter-Action Understanding via Temporal Action Parsing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Qiong Liu,et al.  MV-GNN: Multi-View Graph Neural Network for Compression Artifacts Reduction , 2020, IEEE Transactions on Image Processing.

[15]  Hong Zhu,et al.  Structured Dictionary Learning for Image Denoising Under Mixed Gaussian and Impulse Noise , 2020, IEEE Transactions on Image Processing.

[16]  Guihai Chen,et al.  Shearlet Enhanced Snapshot Compressive Imaging , 2020, IEEE Transactions on Image Processing.

[17]  Shengli Xie,et al.  Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints , 2020, Knowl. Based Syst..

[18]  Tao Zhou,et al.  Multiview Latent Space Learning With Feature Redundancy Minimization , 2020, IEEE Transactions on Cybernetics.

[19]  Rong Wang,et al.  Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding , 2020, Inf. Fusion.

[20]  Witold Pedrycz,et al.  A Weighted Fidelity and Regularization-Based Method for Mixed or Unknown Noise Removal From Images on Graphs , 2020, IEEE Transactions on Image Processing.

[21]  Zenglin Xu,et al.  Auto-weighted multi-view clustering via deep matrix decomposition , 2020, Pattern Recognit..

[22]  Peter Wonka,et al.  Image2StyleGAN++: How to Edit the Embedded Images? , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Sabine Süsstrunk,et al.  Blind Universal Bayesian Image Denoising With Gaussian Noise Level Learning , 2019, IEEE Transactions on Image Processing.

[24]  Kuan-Wen Chen,et al.  FADE: Feature Aggregation for Depth Estimation With Multi-View Stereo , 2020, IEEE Transactions on Image Processing.

[25]  Feiping Nie,et al.  Discriminative and Uncorrelated Feature Selection With Constrained Spectral Analysis in Unsupervised Learning , 2020, IEEE Transactions on Image Processing.

[26]  Zheng Shou,et al.  Deep Tensor ADMM-Net for Snapshot Compressive Imaging , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Xinwang Liu,et al.  Efficient and Effective Incomplete Multi-View Clustering , 2019, AAAI.

[28]  Hong Liu,et al.  Unified Embedding Alignment with Missing Views Inferring for Incomplete Multi-View Clustering , 2019, AAAI.

[29]  Bernard Ghanem,et al.  𝓁0TV: A Sparse Optimization Method for Impulse Noise Image Restoration , 2018, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Wu-Jun Li,et al.  Discrete Latent Factor Model for Cross-Modal Hashing , 2017, IEEE Transactions on Image Processing.

[31]  Xiaodong Wang,et al.  LS-Decomposition for Robust Recovery of Sensory Big Data , 2018, IEEE Transactions on Big Data.

[32]  Songcan Chen,et al.  Doubly Aligned Incomplete Multi-view Clustering , 2018, IJCAI.

[33]  Hong Yang,et al.  Recommendation with Multi-Source Heterogeneous Information , 2018, IJCAI.

[34]  Zenglin Xu,et al.  Self-weighted Multiple Kernel Learning for Graph-based Clustering and Semi-supervised Classification , 2018, IJCAI.

[35]  Guihua Zeng,et al.  Fast first-photon ghost imaging , 2018, Scientific Reports.

[36]  Xuelong Li,et al.  Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification , 2018, IEEE Transactions on Image Processing.

[37]  Pan Zhou,et al.  Tensor Factorization for Low-Rank Tensor Completion , 2018, IEEE Transactions on Image Processing.

[38]  Jianzhong Wang,et al.  Unsupervised feature selection by regularized matrix factorization , 2018, Neurocomputing.

[39]  Antonin Chambolle,et al.  Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications , 2017, SIAM J. Optim..

[40]  Nathan Srebro,et al.  Implicit Regularization in Matrix Factorization , 2017, 2018 Information Theory and Applications Workshop (ITA).

[41]  Bernhard Schölkopf,et al.  Discriminative Transfer Learning for General Image Restoration , 2017, IEEE Transactions on Image Processing.

[42]  Shuqiang Jiang,et al.  A Delicious Recipe Analysis Framework for Exploring Multi-Modal Recipes with Various Attributes , 2017, ACM Multimedia.

[43]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[44]  Qinghua Hu,et al.  Latent Multi-view Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Ahmed M. Elgammal,et al.  Link the Head to the "Beak": Zero Shot Learning from Noisy Text Description at Part Precision , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Qi Tian,et al.  Robust ImageGraph: Rank-Level Feature Fusion for Image Search , 2017, IEEE Transactions on Image Processing.

[47]  Jiajun Zhang,et al.  Implicit Discourse Relation Recognition for English and Chinese with Multiview Modeling and Effective Representation Learning , 2017, ACM Trans. Asian Low Resour. Lang. Inf. Process..

[48]  Pan Zhou,et al.  Bilevel Model-Based Discriminative Dictionary Learning for Recognition , 2017, IEEE Transactions on Image Processing.

[49]  Gang Wang,et al.  An Empirical Study of Language CNN for Image Captioning , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[50]  Yun Fu,et al.  Incomplete Multi-Modal Visual Data Grouping , 2016, IJCAI.

[51]  Wei Liu,et al.  Multi-Modal Curriculum Learning for Semi-Supervised Image Classification , 2016, IEEE Transactions on Image Processing.

[52]  Philip S. Yu,et al.  Multiple Incomplete Views Clustering via Weighted Nonnegative Matrix Factorization with L2, 1 Regularization , 2015, ECML/PKDD.

[53]  Shao-Yuan Li,et al.  Partial Multi-View Clustering , 2014, AAAI.

[54]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[55]  Zhiwu Lu,et al.  Unified Constraint Propagation on Multi-View Data , 2013, AAAI.

[56]  Qiang Ji,et al.  Constrained Clustering and Its Application to Face Clustering in Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[58]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[59]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[60]  Dapeng Wu,et al.  Prediction of Transmission Distortion for Wireless Video Communication: Analysis , 2012, IEEE Transactions on Image Processing.

[61]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Wu-Jun Li,et al.  Relation regularized matrix factorization , 2009, IJCAI 2009.

[63]  Jiawei Han,et al.  Sparse Projections over Graph , 2008, AAAI.

[64]  Derek Greene,et al.  Practical solutions to the problem of diagonal dominance in kernel document clustering , 2006, ICML.

[65]  Jue Wang,et al.  A new maximum margin algorithm for one-class problems and its boosting implementation , 2005, Pattern Recognit..

[66]  Danny C. Sorensen,et al.  The Sylvester equation and approximate balanced reduction , 2002 .

[67]  Majid Mirmehdi,et al.  Experiments on High Resolution Images Towards Outdoor Scene Classification , 2002 .

[68]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[69]  Martin Vetterli,et al.  Wavelet thresholding for multiple noisy image copies , 2000, IEEE Trans. Image Process..

[70]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[71]  Bernhard Schölkopf,et al.  Regularized Principal Manifolds , 1999, J. Mach. Learn. Res..

[72]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[73]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[74]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[75]  M. Hestenes Multiplier and gradient methods , 1969 .

[76]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .