Nonlinear non-negative matrix factorization using deep learning

In this paper, we describe the deep learning method to reduce the dimension of the data samples under the framework Non-negative Matrix Factorization (NMF). That is to say, we try to find the good representation of the data samples for the task of NMF. To this end, a nonlinear NMF optimization model is constructed and the optimization algorithm is developed. The experimental results on some benchmark dataset show the nonlinear dimension reduction helps the NMF to improve the clustering performance.

[1]  Pham The Bao,et al.  An overview of kernel based nonnegative matrix factorization , 2014, 2014 International Conference on Orange Technologies.

[2]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[3]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Ievgen Redko,et al.  Sparsity analysis of learned factors in Multilayer NMF , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[5]  Bao-Gang Hu,et al.  Pairwise Constraints-Guided Non-negative Matrix Factorization for Document Clustering , 2007, IEEE/WIC/ACM International Conference on Web Intelligence (WI'07).

[6]  DeLiang Wang,et al.  Deep neural networks for estimating speech model activations , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[8]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[9]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[10]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[11]  Nam Soo Kim,et al.  NMF-based Target Source Separation Using Deep Neural Network , 2015, IEEE Signal Processing Letters.

[12]  Wanquan Liu,et al.  An efficient nonnegative matrix factorization approach in flexible kernel space , 2009, IJCAI 2009.

[13]  Geoffrey E. Hinton,et al.  Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure , 2007, AISTATS.

[14]  Wei Wang,et al.  Deep Embedding Network for Clustering , 2014, 2014 22nd International Conference on Pattern Recognition.

[15]  Weiyi Meng,et al.  Improving Performance of Web Services Query Matchmaking with Automated Knowledge Acquisition , 2007, IEEE/WIC/ACM International Conference on Web Intelligence (WI'07).

[16]  Mingyi Hong,et al.  Combining sparse NMF with deep neural network: A new classification-based approach for speech enhancement , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[18]  Soo-Young Lee,et al.  Color image processing based on Nonnegative Matrix Factorization with Convolutional Neural Network , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[19]  Jiwen Lu,et al.  Single Sample Face Recognition via Learning Deep Supervised Autoencoders , 2015, IEEE Transactions on Information Forensics and Security.

[20]  Hakan Erdogan,et al.  Deep neural networks for single channel source separation , 2013, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Björn W. Schuller,et al.  Memory-Enhanced Neural Networks and NMF for Robust ASR , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[23]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[24]  Takehito Utsuro,et al.  Novel approach to separation of musical signal sources by NMF , 2014, 2014 12th International Conference on Signal Processing (ICSP).