Computation with Spikes in a Winner-Take-All Network

The winner-take-all (WTA) computation in networks of recurrently connected neurons is an important decision element of many models of cortical processing. However, analytical studies of the WTA performance in recurrent networks have generally addressed rate-based models. Very few have addressed networks of spiking neurons, which are relevant for understanding the biological networks themselves and also for the development of neuromorphic electronic neurons that commmunicate by action potential like address-events. Here, we make steps in that direction by using a simplified Markov model of the spiking network to examine analytically the ability of a spike-based WTA network to discriminate the statistics of inputs ranging from stationary regular to nonstationary Poisson events. Our work extends previous theoretical results showing that a WTA recurrent network receiving regular spike inputs can select the correct winner within one interspike interval. We show first for the case of spike rate inputs that input discrimination and the effects of self-excitation and inhibition on this discrimination are consistent with results obtained from the standard rate-based WTA models. We also extend this discrimination analysis of spiking WTAs to nonstationary inputs with time-varying spike rates resembling statistics of real-world sensory stimuli. We conclude that spiking WTAs are consistent with their continuous counterparts for steady-state inputs, but they also exhibit high discrimination performance with nonstationary inputs.

[1]  Tobi Delbrück,et al.  Modeling orientation selectivity using a neuromorphic multi-chip system , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[2]  Dezhe Z Jin,et al.  Fast computation with spikes in a recurrent neural network. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[4]  Gert Cauwenberghs,et al.  A Multichip Neuromorphic System for Spike-Based Visual Information Processing , 2007, Neural Computation.

[5]  Shun-ichi Amari,et al.  Competitive and Cooperative Aspects in Dynamics of Neural Excitation and Self-Organization , 1982 .

[6]  Tobi Delbrück,et al.  AER Building Blocks for Multi-Layer Multi-Chip Neuromorphic Vision Systems , 2005, NIPS.

[7]  Vittorio Dante,et al.  A software-hardware selective attention system , 2004, Neurocomputing.

[8]  Shih-Chii Liu,et al.  A Normalizing aVLSI Network with Controllable Winner-Take-All Properties , 2002 .

[9]  Shih-Chii Liu.  A Winner-Take-All Circuit with Controllable Soft Max Property , 1999, NIPS.

[10]  Philipp Häfliger,et al.  A time domain winner-take-all network of integrate-and-fire neurons , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[11]  Giacomo Indiveri,et al.  Winner-Take-All Networks with Lateral Excitation , 1997 .

[12]  Janusz A. Starzyk,et al.  CMOS current mode winner-take-all circuit with both excitatory and inhibitory feedback , 1993 .

[13]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[14]  Yingxue Wang,et al.  Quantification of a Spike-Based Winner-Take-All VLSI Network , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  John A. Barnden,et al.  Temporal winner-take-all networks: a time-based mechanism for fast selection in neural networks , 1993, IEEE Trans. Neural Networks.

[16]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[17]  Marinus Maris,et al.  A strong winner-take-all neural network in analogue hardware , 1998 .

[18]  Samuel Kaski,et al.  Winner-take-all networks for physiological models of competitive learning , 1994, Neural Networks.

[19]  Fritz Wysotzki,et al.  Discrimination networks for maximum selection , 2004, Neural Networks.

[20]  Richard Hans Robert Hahnloser,et al.  Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit , 2000, Nature.

[21]  Shih-Chii Liu,et al.  Quantifying Input and Output Spike Statistics of a Winner-Take-All Network in a Vision System , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[22]  Alan L. Yuille,et al.  A Winner-Take-All Mechanism Based on Presynaptic Inhibition Feedback , 1989, Neural Computation.

[23]  Sue L. Denham,et al.  Abstract Stimulus-Specific Adaptation Models , 2011, Neural Computation.

[24]  Bernabé Linares-Barranco,et al.  A modular current-mode high-precision winner-take-all circuit , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[25]  C. Koch,et al.  Recurrent excitation in neocortical cells , 1995 .

[26]  C. Koch,et al.  Recurrent excitation in neocortical circuits , 1995, Science.

[27]  Philipp Häfliger.  Adaptive WTA With an Analog VLSI Neuromorphic Learning Chip , 2007, IEEE Transactions on Neural Networks.

[28]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[29]  Edgar Sanchez-Sinencio,et al.  Min-net winner-take-all CMOS implementation , 1993 .

[30]  Bertram E. Shi,et al.  Neuromorphic implementation of orientation hypercolumns , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[31]  Giacomo Indiveri,et al.  An event-based VLSI network of integrate-and-fire neurons , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[32]  Erik D. Lumer,et al.  Effects of Spike Timing on Winner-Take-All Competition in Model Cortical Circuits , 2000, Neural Computation.

[33]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[34]  Yuguang Fang,et al.  Dynamics of a Winner-Take-All Neural Network , 1996, Neural Networks.

[35]  DeLiang Wang,et al.  Object selection based on oscillatory correlation , 1999, Neural Networks.

[36]  Ernst Niebur,et al.  A Competitive Network of Spiking VLSI Neurons , 2001 .

[37]  Giacomo Indiveri,et al.  A Current-Mode Hysteretic Winner-take-all Network, with Excitatory and Inhibitory Coupling , 2001 .

[38]  Bernabé Linares-Barranco,et al.  A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[39]  Giacomo Indiveri,et al.  A NEUROMORPHIC SELECTIVE ATTENTION ARCHITECTURE WITH DYNAMIC SYNAPSES AND INTEGRATE-AND-FIRE NEURONS , 2004 .

[40]  Martin A. Giese,et al.  Biophysiologically Plausible Implementations of the Maximum Operation , 2002, Neural Computation.

[41]  Richard Granger,et al.  A cortical model of winner-take-all competition via lateral inhibition , 1992, Neural Networks.

[42]  Bard Ermentrout,et al.  Complex dynamics in winner-take-all neural nets with slow inhibition , 1992, Neural Networks.

[43]  John Lazzaro,et al.  Winner-Take-All Networks of O(N) Complexity , 1988, NIPS.

[44]  Wolfgang Maass,et al.  Neural Computation with Winner-Take-All as the Only Nonlinear Operation , 1999, NIPS.

[45]  J Gautrais,et al.  Rate coding versus temporal order coding: a theoretical approach. , 1998, Bio Systems.

[46]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[47]  S. Thorpe,et al.  Surfing a spike wave down the ventral stream , 2002, Vision Research.