Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks

Next-generation wireless networks are witnessing an increasing number of clustering applications, and produce a large amount of non-linear and unlabeled data. In some degree, single kernel methods face the challenging problem of kernel choice. To overcome this problem for non-linear data clustering, multiple kernel graph-based clustering (MKGC) has attracted intense attention in recent years. However, existing MKGC methods suffer from two common problems: (1) they mainly aim to learn a consensus kernel from multiple candidate kernels, slight affinity graph learning, such that cannot fully exploit the underlying graph structure of non-linear data; (2) they disregard the high-order correlations between all base kernels, which cannot fully capture the consistent and complementary information of all kernels. In this paper, we propose a novel non-negative matrix factorization (NMF) tailored graph tensor MKGC method for non-linear data clustering, namely TMKGC. Specifically, TMKGC integrates NMF and graph learning together in kernel space so as to learn multiple candidate affinity graphs. Afterwards, the high-order structure information of all candidate graphs is captured in a 3-order tensor kernel space by introducing tensor singular value decomposition based tensor nuclear norm, such that an optimal affinity graph can be obtained subsequently. Based on the alternating direction method of multipliers, the effective local and distributed solvers are elaborated to solve the proposed objective function. Extensive experiments have demonstrated the superiority of TMKGC compared to the state-of-the-art MKGC methods.

[1]  H. Vincent Poor,et al.  Wireless Communications for Collaborative Federated Learning , 2020, IEEE Communications Magazine.

[2]  Zenglin Xu,et al.  Self-weighted Multiple Kernel Learning for Graph-based Clustering and Semi-supervised Classification , 2018, IJCAI.

[3]  Xinwang Liu,et al.  Multiple Kernel Clustering With Neighbor-Kernel Subspace Segmentation , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Mehdi Bennis,et al.  Wireless Network Intelligence at the Edge , 2018, Proceedings of the IEEE.

[5]  Yuan Sun,et al.  Joint correntropy metric weighting and block diagonal regularizer for robust multiple kernel subspace clustering , 2019, Inf. Sci..

[6]  Zhenwen Ren,et al.  Simultaneous Global and Local Graph Structure Preserving for Multiple Kernel Clustering , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Zhenwen Ren,et al.  Multiple kernel subspace clustering with local structural graph and low-rank consensus kernel learning , 2020, Knowl. Based Syst..

[8]  Zhenwen Ren,et al.  Consensus Affinity Graph Learning for Multiple Kernel Clustering , 2020, IEEE Transactions on Cybernetics.

[9]  Mehdi Bennis,et al.  Communication Efficient Framework for Decentralized Machine Learning , 2020, 2020 54th Annual Conference on Information Sciences and Systems (CISS).

[10]  Lei Wang,et al.  Multiple Kernel k-Means with Incomplete Kernels , 2017, AAAI.

[11]  Mehdi Bennis,et al.  Intelligent Edge: Leveraging Deep Imitation Learning for Mobile Edge Computation Offloading , 2020, IEEE Wireless Communications.

[12]  En Zhu,et al.  Multi-view Clustering via Late Fusion Alignment Maximization , 2019, IJCAI.

[13]  Yung-Yu Chuang,et al.  Multiple Kernel Fuzzy Clustering , 2012, IEEE Transactions on Fuzzy Systems.

[14]  Lei Shi,et al.  Robust Multiple Kernel K-means Using L21-Norm , 2015, IJCAI.

[15]  Zenglin Xu,et al.  Low-rank kernel learning for graph-based clustering , 2019, Knowl. Based Syst..

[16]  Lei Wang,et al.  Multiple Kernel Clustering with Local Kernel Alignment Maximization , 2016, IJCAI.

[17]  Xiaochun Cao,et al.  Low-Rank Tensor Constrained Multiview Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Zenglin Xu,et al.  Structure Learning with Similarity Preserving , 2019, Neural Networks.

[19]  Mehdi Bennis,et al.  Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Ignacio Bosch,et al.  Machine Learning Prediction Approach to Enhance Congestion Control in 5G IoT Environment , 2019, Electronics.

[21]  Shuicheng Yan,et al.  Tensor Low-Rank Representation for Data Recovery and Clustering , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ligang Liu,et al.  Mesh Denoising Guided by Patch Normal Co-Filtering via Kernel Low-Rank Recovery , 2019, IEEE Transactions on Visualization and Computer Graphics.

[23]  Xinwang Liu,et al.  Multiple Kernel Clustering With Global and Local Structure Alignment , 2018, IEEE Access.

[24]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[25]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[26]  Yung-Yu Chuang,et al.  Affinity aggregation for spectral clustering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[28]  Zenglin Xu,et al.  Unified Spectral Clustering with Optimal Graph , 2017, AAAI.

[29]  Nikos D. Sidiropoulos,et al.  A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization , 2015, IEEE Transactions on Signal Processing.

[30]  Tao Mei,et al.  Subspace Clustering by Block Diagonal Representation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Hamido Fujita,et al.  A study of graph-based system for multi-view clustering , 2019, Knowl. Based Syst..

[32]  Franco Davoli,et al.  A Multi-Clustering Approach to Scale Distributed Tenant Networks for Mobile Edge Computing , 2019, IEEE Journal on Selected Areas in Communications.

[33]  Walid Saad,et al.  An Online Framework for Ephemeral Edge Computing in the Internet of Things , 2020, ArXiv.

[34]  Hongbin Zha,et al.  Unified Graph and Low-Rank Tensor Learning for Multi-View Clustering , 2020, AAAI.

[35]  Yan Zhang,et al.  Flexible Auto-Weighted Local-Coordinate Concept Factorization: A Robust Framework for Unsupervised Clustering , 2019, IEEE Transactions on Knowledge and Data Engineering.