Evolutionary nonnegative matrix factorization with adaptive control of cluster quality

Nonnegative matrix factorization (NMF) approximates a given data matrix using linear combinations of a small number of nonnegative basis vectors, weighted by nonnegative encoding coefficients. This enables the exploration of the cluster structure of the data through the examination of the values of the encoding coefficients and therefore, NMF is often used as a popular tool for clustering analysis. However, its encoding coefficients do not always reveal a satisfactory cluster structure. To improve its effectiveness, a novel evolutionary strategy is proposed here to drive the iterative updating scheme of NMF and generate encoding coefficients of higher quality that are capable of offering more accurate and sharper cluster structures. The proposed hybridization procedure that relies on multiple initializations reinforces the robustness of the solution. Additionally, three evolving rules are designed to simultaneously boost the cluster quality and the reconstruction error during the iterative updates. Any clustering performance measure, such as either an internal one relying on the data itself or an external based on the availability of ground truth information, can be employed to drive the evolving procedure. The effectiveness of the proposed method is demonstrated via careful experimental designs and thorough comparative analyses using multiple benchmark datasets.

[1]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[2]  Changshui Zhang,et al.  Efficient Nonnegative Matrix Factorization via projected Newton method , 2012, Pattern Recognit..

[3]  Tao Wu,et al.  Automated Graph Regularized Projective Nonnegative Matrix Factorization for Document Clustering , 2014, IEEE Transactions on Cybernetics.

[4]  Yuan Gao,et al.  Improving molecular cancer class discovery through sparse non-negative matrix factorization , 2005 .

[5]  Hui Xiong,et al.  SAIL: Summation-bAsed Incremental Learning for Information-Theoretic Text Clustering , 2013, IEEE Transactions on Cybernetics.

[6]  Zheng Liu,et al.  Orthogonal Projective Sparse Coding for image representation , 2016, Neurocomputing.

[7]  Jiming Liu,et al.  Improving POMDP Tractability via Belief Compression and Clustering , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Stefan M. Wild.  Seeding Non-Negative Matrix Factorizations with the Spherical K-Means Clustering , 2003 .

[9]  Yao Zhao,et al.  Topographic NMF for Data Representation , 2014, IEEE Transactions on Cybernetics.

[10]  Guoli Wang,et al.  LS-NMF: A modified non-negative matrix factorization algorithm utilizing uncertainty estimates , 2006, BMC Bioinformatics.

[11]  Haifeng Zhao,et al.  A sparse nonnegative matrix factorization technique for graph matching problems , 2014, Pattern Recognit..

[12]  Hui Xiong,et al.  SAIL: summation-based incremental learning for information-theoretic clustering , 2008, KDD.

[13]  Meng Wang,et al.  Image-Based Three-Dimensional Human Pose Recovery by Multiview Locality-Sensitive Sparse Retrieval , 2015, IEEE Transactions on Industrial Electronics.

[14]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[15]  Bernt Schiele,et al.  Introducing a weighted non-negative matrix factorization for image classification , 2003, Pattern Recognit. Lett..

[16]  Reza Boostani,et al.  An Efficient Initialization Method for Nonnegative Matrix Factorization , 2011 .

[17]  Serhat Selcuk Bucak,et al.  Incremental subspace learning via non-negative matrix factorization , 2009, Pattern Recognit..

[18]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[19]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[20]  Seungjin Choi,et al.  Nonnegative Matrix Factorization with Orthogonality Constraints , 2010, J. Comput. Sci. Eng..

[21]  Domingo Giménez,et al.  Parameterized Schemes of Metaheuristics: Basic Ideas and Applications With Genetic Algorithms, Scatter Search, and GRASP , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Anastasios Tefas,et al.  Projected Gradients for Subclass Discriminant Nonnegative Subspace Learning , 2014, IEEE Transactions on Cybernetics.

[23]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[24]  C. D. Meyer,et al.  Initializations for the Nonnegative Matrix Factorization , 2006 .

[25]  Xin-She Yang,et al.  Firefly algorithm, stochastic test functions and design optimisation , 2010, Int. J. Bio Inspired Comput..

[26]  Jun Yu,et al.  Multi-view ensemble manifold regularization for 3D object recognition , 2015, Inf. Sci..

[27]  Jian-Huang Lai,et al.  Extracting non-negative basis images using pixel dispersion penalty , 2012, Pattern Recognit..

[28]  C. L. Philip Chen,et al.  A Multiple-Kernel Fuzzy C-Means Algorithm for Image Segmentation , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Michael Lindenbaum,et al.  Nonnegative Matrix Factorization with Earth Mover's Distance Metric for Image Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Tingting Mu,et al.  Evolutionary Nonnegative Matrix Factorization for Data Compression , 2015, ICIC.

[32]  Hui Xiong,et al.  Understanding and Enhancement of Internal Clustering Validation Measures , 2013, IEEE Transactions on Cybernetics.

[33]  Xinhe Xu,et al.  Facial expression recognition based on PCA and NMF , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[34]  Francisco Tirado,et al.  bioNMF: a versatile tool for non-negative matrix factorization in biology , 2006, BMC Bioinformatics.

[35]  Markus Flierl,et al.  Graph-Preserving Sparse Nonnegative Matrix Factorization With Application to Facial Expression Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Camille Roth,et al.  Natural Scales in Geographical Patterns , 2017, Scientific Reports.

[37]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[38]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[39]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[40]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[41]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[43]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[44]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[45]  Aidong Zhang,et al.  Cluster analysis for gene expression data: a survey , 2004, IEEE Transactions on Knowledge and Data Engineering.

[46]  Jianping Fan,et al.  Multi-View Concept Learning for Data Representation , 2015, IEEE Transactions on Knowledge and Data Engineering.

[47]  R. Schachtner.  Extensions of non-negative matrix factorization and their application to the analysis of wafer test data , 2010 .

[48]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[49]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[50]  Anastasios Tefas,et al.  Subclass discriminant Nonnegative Matrix Factorization for facial image analysis , 2012, Pattern Recognit..

[51]  E. Forgy,et al.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[52]  James M. Keller,et al.  A Cluster Validity Framework Based on Induced Partition Dissimilarity , 2013, IEEE Transactions on Cybernetics.

[53]  Jie Yang,et al.  Initialization enhancer for non-negative matrix factorization , 2007, Eng. Appl. Artif. Intell..