Nonnegative Matrix Factorizations for Intelligent Data Analysis

We discuss nonnegative matrix factorization (NMF) techniques from the point of view of intelligent data analysis (IDA), i.e., the intelligent application of human expertize and computational models for advanced data analysis. As IDA requires human involvement in the analysis process, the understandability of the results coming from computational models has a prominent importance. We therefore review the latest developments of NMF that try to fulfill the understandability requirement in several ways. We also describe a novel method to decompose data into user-defined—hence understandable—parts by means of a mask on the feature matrix, and show the method’s effectiveness through some numerical examples.

[1]  Nicolas Gillis,et al.  Using underapproximations for sparse nonnegative matrix factorization , 2009, Pattern Recognit..

[2]  Mark D. Plumbley.  Conditions for nonnegative independent component analysis , 2002, IEEE Signal Processing Letters.

[3]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[4]  U. Fayyad.  Knowledge Discovery and Data Mining: An Overview , 1995 .

[5]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[6]  Frank Klawonn,et al.  Guide to Intelligent Data Analysis - How to Intelligently Make Sense of Real Data , 2010, Texts in Computer Science.

[7]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[8]  Andrei Yu. Zinovyev,et al.  Blind source separation methods for deconvolution of complex signals in cancer biology , 2013, Biochemical and biophysical research communications.

[9]  Stan Z. Li,et al.  Local non-negative matrix factorization as a visual representation , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[10]  Zhong-Yuan Zhang,et al.  Nonnegative Matrix Factorization: Models, Algorithms and Applications , 2012 .

[11]  Jian Yu,et al.  Semi-supervised Clustering via Constrained Symmetric Non-negative Matrix Factorization , 2012, Brain Informatics.

[12]  Corrado Mencar,et al.  Part-Based Data Analysis with Masked Non-negative Matrix Factorization , 2014, ICCSA.

[13]  Michel C. Desmarais.  Conditions for Effectively Deriving a Q-Matrix from Data with Non-negative Matrix Factorization. Best Paper Award , 2011, EDM.

[14]  Antonio J. Plaza,et al.  A Signal Processing Perspective on Hyperspectral Unmixing: Insights from Remote Sensing , 2014, IEEE Signal Processing Magazine.

[15]  Weiyi Meng,et al.  Improving Performance of Web Services Query Matchmaking with Automated Knowledge Acquisition , 2007, IEEE/WIC/ACM International Conference on Web Intelligence (WI'07).

[16]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[17]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Jing Hua,et al.  Non-negative matrix factorization for semi-supervised data clustering , 2008, Knowledge and Information Systems.

[19]  H. Gulliksen.  Theory of mental tests , 1952 .

[20]  Nicolas Gillis,et al.  Two algorithms for orthogonal nonnegative matrix factorization with application to clustering , 2012, Neurocomputing.

[21]  Constantine Kotropoulos,et al.  Applying Supervised Classifiers Based on Non-negative Matrix Factorization to Musical Instrument Classification , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[22]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[23]  Seungjin Choi,et al.  Semi-Supervised Nonnegative Matrix Factorization , 2010, IEEE Signal Processing Letters.

[24]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[25]  Nicolas Gillis,et al.  The Why and How of Nonnegative Matrix Factorization , 2014, ArXiv.

[26]  Fei Wang,et al.  Community discovery using nonnegative matrix factorization , 2011, Data Mining and Knowledge Discovery.

[27]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[28]  Slim Essid,et al.  Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring , 2013, IEEE Transactions on Multimedia.

[29]  Mikhail Kotov,et al.  Non-negative matrix factorization with linear constraints for single-channel speech enhancement , 2013, INTERSPEECH.

[30]  C. Spearman.  General intelligence Objectively Determined and Measured , 1904 .

[31]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[32]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[33]  Inderjit S. Dhillon,et al.  Generalized Nonnegative Matrix Approximations with Bregman Divergences , 2005, NIPS.

[34]  Seungjin Choi,et al.  Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds , 2010, Inf. Process. Manag..

[35]  P. Hopke.  Receptor modeling in environmental chemistry , 1985 .

[36]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[37]  Ziqiang Wang,et al.  Face Recognition Based on NMF and SVM , 2009, 2009 Second International Symposium on Electronic Commerce and Security.

[38]  C. Theekapun,et al.  Facial Expression Recognition Based on , 2008 .

[39]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[40]  최승진.  Algorithms for orthogonal nonnegative matrix factorization , 2018 .

[41]  Nicolas Gillis,et al.  Hierarchical Clustering of Hyperspectral Images Using Rank-Two Nonnegative Matrix Factorization , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Amnon Shashua,et al.  Nonnegative Sparse PCA , 2006, NIPS.

[43]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[44]  Massimo Minervini,et al.  Nonnegative Matrix Factorizations Performing Object Detection and Localization , 2012, Appl. Comput. Intell. Soft Comput..

[45]  Christoph Schnörr,et al.  Learning Sparse Representations by Non-Negative Matrix Factorization and Sequential Cone Programming , 2006, J. Mach. Learn. Res..

[46]  Yanhua Chen,et al.  Non-Negative Matrix Factorization for Semisupervised Heterogeneous Data Coclustering , 2010, IEEE Transactions on Knowledge and Data Engineering.

[47]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[48]  G. Stewart,et al.  A generalization of the Eckart-Young-Mirsky matrix approximation theorem , 1987 .

[49]  Erkki Oja,et al.  A "nonnegative PCA" algorithm for independent component analysis , 2004, IEEE Transactions on Neural Networks.

[50]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Konstantinos Drakakis,et al.  Analysis of Financial Data Using , 2008 .

[52]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[53]  Bernardete Ribeiro,et al.  Extracting Discriminative Features Using Non-negative Matrix Factorization in Financial Distress Data , 2009, ICANNGA.

[54]  Constantine Kotropoulos,et al.  Musical Instrument Classification using Non-Negative Matrix Factorization Algorithms and Subset Feature Selection , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[55]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[56]  Junpeng Chen,et al.  Topic sense induction from social tags based on non-negative matrix factorization , 2014, Inf. Sci..

[57]  Mark D. Plumbley.  Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[58]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[59]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[60]  David J. Hand,et al.  Intelligent Data Analysis: Issues and Opportunities , 1998, Intell. Data Anal..

[61]  Francisco Tirado,et al.  bioNMF: a web-based tool for nonnegative matrix factorization in biology , 2008, Nucleic Acids Res..

[62]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[63]  I. Jolliffe.  Principal Component Analysis , 2002 .

[64]  Nicoletta Del Buono.  A Penalty Function for Computing Orthogonal Non-negative Matrix Factorizations , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[65]  Parag Kulkarni,et al.  A Survey of Semi-Supervised Learning Methods , 2008, 2008 International Conference on Computational Intelligence and Security.

[66]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[67]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[69]  Martin J. McKeown,et al.  Functional Segmentation of fMRI Data Using Adaptive Non-negative Sparse PCA (ANSPCA) , 2009, MICCAI.

[70]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[71]  Jordi Vitrià,et al.  Evaluation of distance metrics for recognition based on non-negative matrix factorization , 2003, Pattern Recognit. Lett..

[72]  Yunde Jia,et al.  FISHER NON-NEGATIVE MATRIX FACTORIZATION FOR LEARNING LOCAL FEATURES , 2004 .

[73]  Aapo Hyvärinen,et al.  Survey on Independent Component Analysis , 1999 .

[74]  Frederic M. Lord.  THE RELATION OF TEST SCORE TO THE TRAIT UNDERLYING THE TEST , 1952 .

[75]  Andrea Montanari,et al.  Non-Negative Principal Component Analysis: Message Passing Algorithms and Sharp Asymptotics , 2014, IEEE Transactions on Information Theory.

[76]  Michel C. Desmarais,et al.  Item to Skills Mapping: Deriving a Conjunctive Q-matrix from Data , 2012, ITS.

[77]  Hongtao Lu,et al.  Semi-supervised non-negative matrix factorization for image clustering with graph Laplacian , 2013, Multimedia Tools and Applications.

[78]  Shuicheng Yan,et al.  Non-Negative Semi-Supervised Learning , 2009, AISTATS.

[79]  Corrado Mencar,et al.  Subtractive clustering for seeding non-negative matrix factorizations , 2014, Inf. Sci..

[80]  C. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and K-means - Spectral Clustering , 2005 .

[81]  Nicolas Gillis,et al.  Sparse nonnegative matrix underapproximation and its application to hyperspectral image analysis , 2011, 2011 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[82]  P. Hopke,et al.  Source Identification of Atlanta Aerosol by Positive Matrix Factorization , 2003, Journal of the Air & Waste Management Association.

[83]  Richard Bellman,et al.  Adaptive Control Processes - A Guided Tour (Reprint from 1961) , 2015, Princeton Legacy Library.

[84]  Chengyu Liu,et al.  Biclustering of gene expression data by non-smooth non-negative matrix factorization , 2010 .

[85]  Xinhe Xu,et al.  Facial expression recognition based on PCA and NMF , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[86]  R. Plemmons,et al.  Optimality, computation, and interpretation of nonnegative matrix factorizations , 2004 .

[87]  David J. Hand.  Intelligent Data Analysis: Issues and Opportunities , 1998, Intell. Data Anal..

[88]  Ping Liu,et al.  The application of principal component analysis and non-negative matrix factorization to analyze time-resolved optical waveguide absorption spectroscopy data , 2013 .

[89]  Hongtao Lu,et al.  Non-negative Matrix Factorization with Pairwise Constraints and Graph Laplacian , 2015, Neural Processing Letters.

[90]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[91]  Constantine Kotropoulos,et al.  Comparison of subspace analysis-based and statistical model-based algorithms for musical instrument classification , 2005 .

[92]  Gabriella Casalino.  Non-negative factorization methods for extracting semantically relevant features in Intelligent Data Analysis , 2015 .

[93]  Sen Jia,et al.  A Complexity Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2007, ICA.

[94]  Lawrence K. Saul,et al.  Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction , 2011, ArXiv.

[95]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[96]  Jordi Vitrià,et al.  Non-negative Matrix Factorization for Face Recognition , 2002, CCIA.

[97]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[98]  Chris H. Q. Ding,et al.  Nonnegative Matrix Factorizations for Clustering: A Survey , 2018, Data Clustering: Algorithms and Applications.

[99]  Andri Mirzal,et al.  Clustering and Latent Semantic Indexing Aspects of the Nonnegative Matrix Factorization , 2011, ArXiv.

[100]  Nanning Zheng,et al.  Non-negative matrix factorization for visual coding , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[101]  Hujun Bao,et al.  Understanding the Power of Clause Learning , 2009, IJCAI.

[102]  Karthik Devarajan,et al.  Nonnegative Matrix Factorization: An Analytical and Interpretive Tool in Computational Biology , 2008, PLoS Comput. Biol..

[103]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[104]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[105]  Suvrit Sra,et al.  Sparse nonnegative matrix approximation: new formulations and algorithms , 2010 .

[106]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[107]  John H. Holmes,et al.  Intelligent data analysis in biomedicine , 2007, J. Biomed. Informatics.

[108]  Nanning Zheng,et al.  Non-negative matrix factorization based methods for object recognition , 2004, Pattern Recognit. Lett..

[109]  F. Lord.  A theory of test scores. , 1952 .

[110]  Pierluigi Siano,et al.  A Fuzzy Logic Controller to Increase Fault Ride-Through Capability of Variable Speed Wind Turbines , 2012, Appl. Comput. Intell. Soft Comput..

[111]  Jing Hua,et al.  Incorporating User Provided Constraints into Document Clustering , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[112]  Michael Berthold,et al.  Intelligent Data Analysis , 1999, Springer Berlin Heidelberg.

[113]  Nicolas Gillis,et al.  Dimensionality reduction, classification, and spectral mixture analysis using non-negative underapproximation , 2011 .

[114]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[115]  Haifeng Liu,et al.  Non-Negative Matrix Factorization with Constraints , 2010, AAAI.

[116]  Tao Li,et al.  The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[117]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[118]  Yuan Gao,et al.  Improving molecular cancer class discovery through sparse non-negative matrix factorization , 2005 .

[119]  Paul Geladi,et al.  Hyperspectral Image Data Conditioning and Regression Analysis , 2007 .

[120]  Mark D. Plumbley.  Algorithms for Non-Negative Independent Component Analysis , 2002 .

[121]  Mark D. Plumbley,et al.  BLIND SEPARATION OF POSITIVE SOURCES USING NON-NEGATIVE PC A , 2003 .

[122]  Andrei Doncescu,et al.  Non Negative Matrix Factorization Clustering Capabilities; Application on Multivariate Image Segmentation , 2009, 2009 International Conference on Complex, Intelligent and Software Intensive Systems.

[123]  Yujiu Yang,et al.  Pairwise Constraints-Guided Non-negative Matrix Factorization for Document Clustering , 2007, International Conference on Wirtschaftsinformatik.