Analysis on Wang's kWTA with Stochastic Output Nodes

Recently, a Dual Neural Network-based kWTA has been proposed, in which the output nodes are defined as a Heaviside step activation function. In this paper, we extend this model by considering that the output nodes are stochastic. Precisely, we define this stochastic behavior by the logistic function. It is shown that the DNN-based kWTA with stochastic output nodes is able to converge and the convergence rates of this network are three folds. Finally, the energy function governing the dynamical behavior of the network is unveiled.

[1]  James T. Kwok,et al.  Advances in Neural Networks - ISNN 2010, 7th International Symposium on Neural Networks, ISNN 2010, Shanghai, China, June 6-9, 2010, Proceedings, Part I , 2010, ISNN.

[2]  Jinde Cao,et al.  A Novel Recurrent Neural Network With One Neuron and Finite-Time Convergence for $k$-Winners-Take-All Operation , 2010, IEEE Transactions on Neural Networks.

[3]  Jun Wang,et al.  Analysis and Design of a $k$ -Winners-Take-All Model With a Single State Variable and the Heaviside Step Activation Function , 2010, IEEE Transactions on Neural Networks.

[4]  Corneliu A. Marinov,et al.  Another K-winners-take-all analog neural network , 2000, IEEE Trans. Neural Networks Learn. Syst..

[5]  Lipo Wang,et al.  Noise injection into inputs in sparsely connected Hopfield and winner-take-all neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Xiaolin Hu,et al.  An Improved Dual Neural Network for Solving a Class of Quadratic Programming Problems and Its $k$-Winners-Take-All Application , 2008, IEEE Transactions on Neural Networks.

[7]  Corneliu A. Marinov,et al.  Time-Oriented Synthesis for a WTA Continuous-Time Neural Network Affected by Capacitive Cross-Coupling , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Jun Wang,et al.  Parametric Sensitivity and Scalability of k-Winners-Take-All Networks with a Single State Variable and Infinity-Gain Activation Functions , 2010, ISNN.

[9]  Paul W. Munro,et al.  Nets with Unreliable Hidden Nodes Learn Error-Correcting Codes , 1992, NIPS.

[10]  Yaser S. Abu-Mostafa,et al.  On the K-Winners-Take-All Network , 1988, NIPS.

[11]  Andrew Chi-Sing Leung,et al.  A Fault-Tolerant Regularizer for RBF Networks , 2008, IEEE Transactions on Neural Networks.