Learning Low Dimensional Patterns with Multi-Scale Local Representation

Manifold learning has been extensively applied to the field of data dimensionality reduction and pattern recognition in recent years since the related algorithms can efficiently retrieve the intrinsic geometric features of image data in computational intelligence. In this paper, we propose a new manifold learning model based on the nonnegative matrix factorization. In this model, α-divergence with locally linear representation is adopted as the objective function for nonnegative matrix decomposition for both basis and low dimensional representation or encoding vectors to capture features in high dimensional space. With a variable neighborhood size in the learning, the proposed model can learn the linear features and at the same time learn the local similarity of images in a neighborhood. Test results on JAFFE and ORL image datasets show that the proposed algorithms can obtain the state of art performance on the tasks of image clustering.

[1]  Wai Lok Woo,et al.  Machine Learning Source Separation Using Maximum a Posteriori Nonnegative Matrix Factorization , 2014, IEEE Transactions on Cybernetics.

[2]  Renato D. C. Monteiro,et al.  Group Sparsity in Nonnegative Matrix Factorization , 2012, SDM.

[4]  Chris H. Q. Ding,et al.  On Trivial Solution and Scale Transfer Problems in Graph Regularized NMF , 2011, IJCAI.

[5]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Justin P. Haldar,et al.  Greedy algorithms for nonnegativity-constrained simultaneous sparse recovery , 2016, Signal Process..

[7]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[8]  Jun Zhou,et al.  Multilayer manifold and sparsity constrainted nonnegative matrix factorization for hyperspectral unmixing , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[9]  Zhaojun Bai,et al.  Robust and Efficient Computation of Eigenvectors in a Generalized Spectral Method for Constrained Clustering , 2017, AISTATS.

[10]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[11]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jinhui Tang,et al.  Robust Structured Nonnegative Matrix Factorization for Image Representation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Haesun Park,et al.  Fast rank-2 nonnegative matrix factorization for hierarchical document clustering , 2013, KDD.

[14]  Yuri Owechko,et al.  Zero Shot Learning via Multi-scale Manifold Regularization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Guang Shi,et al.  Graph-based discriminative nonnegative matrix factorization with label information , 2017, Neurocomputing.

[16]  Zenglin Xu,et al.  Manifold regularized matrix completion for multilabel classification , 2016, Pattern Recognit. Lett..

[17]  Jacek M. Zurada,et al.  Deep Learning of Part-Based Representation of Data Using Sparse Autoencoders With Nonnegativity Constraints , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Hassan Ghassemian,et al.  Spectral Unmixing of Hyperspectral Imagery Using Multilayer NMF , 2014, IEEE Geoscience and Remote Sensing Letters.

[19]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[20]  Andrzej Cichocki,et al.  Csiszár's Divergences for Non-negative Matrix Factorization: Family of New Algorithms , 2006, ICA.

[21]  Xuelong Li,et al.  A Class of Manifold Regularized Multiplicative Update Algorithms for Image Clustering , 2015, IEEE Transactions on Image Processing.

[22]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[24]  Carl-Fredrik Westin,et al.  Fast Manifold Learning Based on Riemannian Normal Coordinates , 2005, SCIA.

[25]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[26]  Xavier Bresson,et al.  Robust Principal Component Analysis on Graphs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Zhenyue Zhang,et al.  Low-Rank Matrix Approximation with Manifold Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Jochen J. Steil,et al.  Online learning and generalization of parts-based image representations by non-negative sparse autoencoders , 2012, Neural Networks.

[29]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[30]  Xuelong Li,et al.  Manifold Regularized Sparse NMF for Hyperspectral Unmixing , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Sergio Cruces,et al.  Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization , 2011, Entropy.

[32]  Yuan Yan Tang,et al.  Topology Preserving Non-negative Matrix Factorization for Face Recognition , 2008, IEEE Transactions on Image Processing.

[33]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Aihua Li,et al.  Sparsity-Constrained Deep Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2018, IEEE Geoscience and Remote Sensing Letters.

[35]  Zenglin Xu,et al.  Manifold regularized matrix completion for multi-label learning with ADMM , 2018, Neural Networks.