An Efficient Knowledge Clustering Algorithm

In this chapter, we have proposed a NRCG-ONMF method which alternatively updates the orthogonal factor U by doing nonlinear search on Stiefel manifold, and updates the nonnegative factor V in a coordinate manner with closed form solutions. The convergence of NRCG-ONMF has been analyzed. Our approach sheds lights on an promising way to efficiently perform ONMF and shows great potential to handle large scale problems. We evaluate the proposed method on clustering tasks. Extensive experiments on both synthetic and real-world data sets demonstrate that the proposed NRCG-ONMF method outperforms other ONMF methods in terms of the effectiveness on preservation of orthogonality, optimization efficiency and clustering performance.

[1]  Joydeep Ghosh,et al.  Under Consideration for Publication in Knowledge and Information Systems Generative Model-based Document Clustering: a Comparative Study , 2003 .

[2]  Yu-Jin Zhang,et al.  Nonnegative Matrix Factorization: A Comprehensive Review , 2013, IEEE Transactions on Knowledge and Data Engineering.

[3]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[4]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  Fuming Sun,et al.  Graph regularized and sparse nonnegative matrix factorization with hard constraints for data representation , 2016, Neurocomputing.

[7]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[8]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[10]  Robert P. W. Duin,et al.  Handwritten digit recognition by combined classifiers , 1998, Kybernetika.

[11]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[12]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[13]  Seungjin Choi,et al.  Algorithms for orthogonal nonnegative matrix factorization , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[14]  Xindong Wu,et al.  Nonnegative Matrix Factorization on Orthogonal Subspace , 2010, Pattern Recognit. Lett..

[15]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[16]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[17]  Lina Yao,et al.  Efficient Orthogonal Non-negative Matrix Factorization over Stiefel Manifold , 2016, CIKM.

[18]  Dimitris S. Papailiopoulos,et al.  Orthogonal NMF through Subspace Exploration , 2015, NIPS.

[19]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[20]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[21]  Inderjit S. Dhillon,et al.  Fast coordinate descent methods with variable selection for non-negative matrix factorization , 2011, KDD.

[22]  Feiping Nie,et al.  Robust Capped Norm Nonnegative Matrix Factorization: Capped Norm NMF , 2015, CIKM.

[23]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[24]  Bart Vandereycken,et al.  Low-Rank Matrix Completion by Riemannian Optimization , 2013, SIAM J. Optim..

[25]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[26]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[28]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[29]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[30]  Wotao Yin,et al.  A Curvilinear Search Method for p-Harmonic Flows on Spheres , 2009, SIAM J. Imaging Sci..

[31]  Nicolas Gillis,et al.  Two algorithms for orthogonal nonnegative matrix factorization with application to clustering , 2012, Neurocomputing.

[32]  Luo Si,et al.  Non-Negative Matrix Factorization Clustering on Multiple Manifolds , 2010, AAAI.

[33]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.