A Collaborative Neurodynamic Approach to Symmetric Nonnegative Matrix Factorization

This paper presents a collaborative neurodynamic approach to symmetric nonnegative matrix factorization (SNMF). First, a formulated nonconvex optimization problem of SNMF is described. To solve this problem, a neurodynamic model based on an augmented Lagrangian function is proposed and proven to be convergent to a strict local optimal solution under the second-order sufficiency condition. Next, a group of neurodynamic models are employed to search for an optimal factorized matrix by using particle swarm algorithm to update the initial neuronal states. The efficacy of the proposed approach is substantiated on two datasets.

[1]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[2]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[3]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[4]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[5]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .

[6]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[7]  Wei Xing Zheng,et al.  Discrete-Time Neural Network for Fast Solving Large Linear $L_{1}$ Estimation Problems and its Application to Image Restoration , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Xinyi Le,et al.  A Neurodynamic Optimization Approach to Bilevel Quadratic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[10]  Jun Wang,et al.  Model Predictive Control of Nonlinear Systems With Unmodeled Dynamics Based on Feedforward and Recurrent Neural Networks , 2012, IEEE Transactions on Industrial Informatics.

[11]  Yunong Zhang,et al.  A dual neural network for convex quadratic programming subject to linear equality and inequality constraints , 2002 .

[12]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[13]  Xiaolin Hu,et al.  Convergence of a Recurrent Neural Network for Nonconvex Optimization Based on an Augmented Lagrangian Function , 2007, ISNN.

[14]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[15]  Jun Wang,et al.  A nonnegative matrix factorization algorithm based on a discrete-time projection neural network , 2018, Neural Networks.

[16]  Jun Wang,et al.  A Collective Neurodynamic Approach to Constrained Global Optimization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[18]  Jun Wang,et al.  A one-layer recurrent neural network for support vector machine learning , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Jianchao Fan,et al.  A Collective Neurodynamic Optimization Approach to Nonnegative Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Chuandong Li,et al.  A recurrent neural network for adaptive beamforming and array correction , 2016, Neural Networks.

[21]  Xinyi Le,et al.  A Two-Time-Scale Neurodynamic Approach to Constrained Minimax Optimization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Chris H. Q. Ding,et al.  Symmetric Nonnegative Matrix Factorization for Graph Clustering , 2012, SDM.

[23]  Jun Wang,et al.  A collective neurodynamic optimization approach to bound-constrained nonconvex optimization , 2014, Neural Networks.

[24]  Jun Wang,et al.  A Collaborative Neurodynamic Approach to Multiobjective Optimization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[26]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[27]  Philip S. Yu,et al.  Relational clustering by symmetric convex coding , 2007, ICML '07.

[28]  Philip S. Yu,et al.  Co-clustering by block value decomposition , 2005, KDD '05.

[29]  Qingshan Liu,et al.  A Collaborative Neurodynamic Approach to Multiple-Objective Distributed Optimization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Hanif D. Sherali,et al.  Solutions Manual to accompany Nonlinear Programming: Theory and Algorithms , 2013 .