On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering
Current nonnegative matrix factorization (NMF) deals with X = FG type. We provide a systematic analysis and extensions of NMF to the symmetric W = HH , and the weighted W = HSH . We show that (1) W = HH is equivalent to Kernel K-means clustering and the Laplacian-based spectral clustering. (2) X = FG is equivalent to simultaneous clustering of rows and columns of a bipartite graph. Algorithms are given for computing these symmetric NMFs.
On the Equivalence of Nonnegative Matrix Factorization and K-means - Spectral Clustering
On the Equivalence of Nonnegative Matrix Factorization and K-means — Spectral Clustering Chris Ding ∗ Xiaofeng He ∗ Horst D. Simon ∗ Rong Jin † December 4, 2005 Abstract We provide a systematic analysis of nonnegative matrix factorization (NMF) relating to data cluster- ing. We generalize the usual X = F G T decomposition to the symmetric W = HH T and W = HSH T decompositions. We show that (1) W = HH T is equivalent to Kernel K-means clustering and the Laplacian-based spectral clustering. (2) X = F G T is equivalent to simultaneous clustering of rows and columns of a bipartite graph. We emphasizes the importance of orthogonality in NMF and soft clustering nature of NMF. These results are verified with experiments on face images and newsgroups. Introduction Standard factorization of a data matrix uses singular value decomposition (SVD) as widely used in principal component analysis (PCA). However, for many dataset such as images and text, the original data matrices are nonnegative. A factorization such as SVD contains negative entries and is difficult to interpret for some applications. In contrast, nonnegative matrix factorization (NMF) [18, 19] restricts the entries in matrix factors to be nonnegative. NMF has been shown recently to be useful for many applications in environment [25], chemometrics [29], pattern recognition [20], multimedia [6], text mining [31, 26] and DNA gene expressions [3]. This is also extended to classification [27]. A number of stuides focus on further developing NMF computational methodologies [15, 22, 26, 5, 21]. Let X = (x 1 , . . . , x n ) ∈ R p×n be the data matrix of nonnegative elements. In image processing, each column x i is a 2D array of pixels gray level. In text processing, each column is a document. The NMF factorizes X into two nonnegative matrices, X ≈ F G T , n×k where F = (f 1 , · · · , f k ) ∈ R p×k and G = (g 1 , · · · , g k ) ∈ R + . k is a pre-specified parameter. NMF can be traced back to 1970s (communication from Gene Golub) and has been studied by Paatero [25, 29]. The work of Lee and Seung [18, 19] brought much attention to NMF in machine learning and data mining communities. There appears to have some confusions, however. Lee and Seung emphasizes[18] that NMF factors f k contain coherent parts of the original data (images), for example a nose or an eye. Later experiments [16, 20] do not support the parts-of-whole interpretation of NMF. In fact, Hoyer[16] and Li, et al[20] specifically propose sparsification schemes to achieve the parts-of-whole pictures. ∗ Lawrence † Department Berkeley National Laboratory, University of California, Berkeley, CA 94720. of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824.
Nonnegative Matrix Factorization for Combinatorial Optimization: Spectral Clustering, Graph Matching, and Clique Finding
Nonnegative matrix factorization (NMF) is a versatile model for data clustering. In this paper, we propose several NMF inspired algorithms to solve different data mining problems. They include (1) multi-way normalized cut spectral clustering, (2) graph matching of both undirected and directed graphs, and (3) maximal clique finding on both graphs and bipartite graphs. Key features of these algorithms are (a) they are extremely simple to implement; and (b) they are provably convergent. We conduct experiments to demonstrate the effectiveness of these new algorithms. We also derive a new spectral bound for the size of maximal edge bicliques as a byproduct of our approach.
neural network power system internet of things electric vehicle data analysi renewable energy smart grid learning algorithm power grid image compression hyperspectral image matrix factorization source separation cyber-physical system energy management system sparse representation deep convolutional cloud storage blind source separation demand response blind source gradient method renewable energy system grid system dictionary learning hyperspectral datum latent semantic spectral clustering nonnegative matrix nonnegative matrix factorization hyperspectral imagery low rank image representation image inpainting public cloud matrix completion spectral datum smart grid system smart grid technology remote datum smart grid communication tensor factorization data matrix latent factor future smart grid factorization method spectral unmixing grid communication hyperspectral unmixing international system future smart smart power grid nonnegative matrice power grid system dictionary learning algorithm matrix factorization method data possession projected gradient graph regularized factorization based nonnegative tensor provable data possession system of units image inpainting method smart grid security provable datum public cloud storage matrix factorization technique projected gradient method factorization technique nonnegative tensor factorization nmf algorithm low-rank matrix factorization exemplar-based image inpainting image inpainting technique emerging smart grid matrix factorization problem multiplicative update based image inpainting regularized nonnegative matrix constrained nonnegative matrix sparse nonnegative kernel k-means clustering regularized nonnegative sparse nonnegative matrix matrix and tensor sparse nmf constrained nonnegative high-dimensional vector nmf method orthogonal nonnegative matrix graph regularized nonnegative nonnegative datum multi-way datum nonnegative tucker decomposition lee and seung weighted nonnegative matrix weighted nonnegative robust nonnegative matrix projective nonnegative matrix als algorithm robust nonnegative input data matrix projective nonnegative semantic image inpainting fast nonnegative wind power