A recurrent neural network with a tunable activation function for solving k-winners-take-all

In this paper, a finite time recurrent neural network with a tunable activation function is presented to solve the k-winners-take-all problem. The activation function has two tunable parameters which give more flexibility to design neural network. By Lyapunov theorem, the proposed neural network model can converge to the equilibrium point in finite time. Comparing with the existing neural networks, the faster convergence speed can be obtained. Particularly, proposed neural network has high robustness against noise. The effectiveness of our methods is validated by theoretical analysis and numerical simulations.

[1]  Philip J. Cook,et al.  The Winner-Take-All Society: Why the Few at the Top Get So Much More Than the Rest of Us , 1996 .

[2]  Shuai Li,et al.  Bluetooth aided mobile phone localization , 2014, ACM Trans. Embed. Comput. Syst..

[3]  Xiaolin Hu,et al.  A New Recurrent Neural Network for Solving Convex Quadratic Programming Problems With an Application to the $k$-Winners-Take-All Problem , 2009, IEEE Transactions on Neural Networks.

[4]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[5]  C. Beveridge,et al.  Update on Apical Dominance and Shoot Branching Apical Dominance and Shoot Branching. Divergent Opinions or Divergent Mechanisms? , 2006 .

[6]  Yunong Zhang,et al.  A dual neural network for convex quadratic programming subject to linear equality and inequality constraints , 2002 .

[7]  Magnus Enquist,et al.  Neural networks and animal behavior , 2005 .

[8]  Shuai Li,et al.  Model-free control of Lorenz chaos using an approximate optimal control strategy , 2012 .

[9]  Marco Idiart,et al.  A Second Function of Gamma Frequency Oscillations: An E%-Max Winner-Take-All Mechanism Selects Which Cells Fire , 2009, The Journal of Neuroscience.

[10]  Cristóbal López,et al.  Species competition: coexistence, exclusion and clustering , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[11]  Shuai Li,et al.  Selective Positive–Negative Feedback Produces the Winner-Take-All Competition in Recurrent Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[13]  Jun Wang,et al.  A recurrent neural network with exponential convergence for solving convex quadratic program and related linear piecewise equations , 2004, Neural Networks.

[14]  Qingshan Liu,et al.  Two k-winners-take-all networks with discontinuous activation functions , 2008, Neural Networks.

[15]  T. Robbins,et al.  The neuropsychology of ventral prefrontal cortex: Decision-making and reversal learning , 2004, Brain and Cognition.

[16]  Shuai Li,et al.  Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks , 2012, Neurocomputing.

[17]  Weibing Gao,et al.  Variable structure control of nonlinear systems: a new approach , 1993, IEEE Trans. Ind. Electron..

[18]  John G. Harris,et al.  Noise-Robust Automatic Speech Recognition Using a Predictive Echo State Network , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Anderson,et al.  Controlled competitive dynamics in a photorefractive ring oscillator: "Winner-takes-all" and the "voting-paradox" dynamics. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[20]  X. Xia,et al.  Semi-global finite-time observers for nonlinear systems , 2008, Autom..

[21]  Jean-Jacques E. Slotine,et al.  Fast computation with neural oscillators , 2003, Neurocomputing.

[22]  Mahesan Niranjan,et al.  The use of recurrent neural networks for classification , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[23]  David Dubnau.  Winner takes all in a race for cell fate , 2011, Molecular systems biology.

[24]  Holger Schulze,et al.  Auditory Cortical Contrast Enhancing by Global Winner-Take-All Inhibitory Interactions , 2008, PloS one.

[25]  Dennis S. Bernstein,et al.  Finite-Time Stability of Continuous Autonomous Systems , 2000, SIAM J. Control. Optim..

[26]  Yuehua Huang,et al.  Global finite-time stabilisation for a class of nonlinear systems , 2012, Int. J. Syst. Sci..

[27]  Pavlo V. Tymoshchuk.  A fast analogue K-winners-take-all neural circuit , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[28]  Kate Smith-Miles,et al.  Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research , 1999, INFORMS J. Comput..

[29]  Yangming Li,et al.  A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application , 2013, Neural Networks.

[30]  Changyin Sun,et al.  A novel neural dynamical approach to convex quadratic program and its efficient applications , 2009, Neural Networks.