Hierarchical learning of conjunctive concepts in spiking neural networks

The temporal correlation hypothesis proposes that synchronous activity in different regions of the brain describes integral entities. This temporal binding approach is a possible solution to the longstanding binding problem of representing composite objects. To complement the dynamic nature of temporal binding, a recruitment learning method has been proposed for providing long-term storage. We improve the recruitment method to use a more realistic and powerful spiking neuron model. However, using continuous-time spiking neurons and brain-like connectivity assumptions poses new problems in hierarchical recruitment. First, we propose timing parameter constraints for recruitment over asymmetrically delayed lines. Second, we calculate required feedforward excitatory and lateral inhibitory connection densities for stable propagation of activity independent of network size.

[1]  Cengiz Günay,et al.  Using Temporal Binding for Robust Connectionist Recruitment Learning over Delayed Lines ∗ , 2003 .

[2]  E Bienenstock,et al.  On the dimensionality of cortical graphs , 1996, Journal of Physiology-Paris.

[3]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[4]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[5]  C. Gunay,et al.  The required measures of phase segregation in distributed cortical processing , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[6]  Stefan Wermter,et al.  An Overview of Hybrid Neural Systems , 1998, Hybrid Neural Systems.

[7]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[8]  James A. Anderson,et al.  A simple neural network generating an interactive memory , 1972 .

[9]  Istvan S. N. Berkeley Density Plots of Hidden Value Unit Activations Reveal Interpretable Bands , 1995, Connect. Sci..

[10]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[11]  Timothy Masters,et al.  Multilayer Feedforward Networks , 1993 .

[12]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[13]  Peter L. Bartlett,et al.  Vapnik-Chervonenkis dimension of neural nets , 2003 .

[14]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[15]  C. Koch,et al.  A brief history of time (constants). , 1996, Cerebral cortex.

[16]  Peter König,et al.  Binding by temporal structure in multiple feature domains of an oscillatory neuronal network , 1994, Biological Cybernetics.

[17]  Lokendra Shastri,et al.  Semantic Networks: An Evidential Formalization and Its Connectionist Realization , 1988 .

[18]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[19]  Martin Schneider,et al.  Activity-Dependent Development of Axonal and Dendritic Delays, or, Why Synaptic Transmission Should Be Unreliable , 2002, Neural Computation.

[20]  Antony Browne,et al.  Connectionist inference models , 2001, Neural Networks.

[21]  C. Stevens,et al.  Heterogeneity of Release Probability, Facilitation, and Depletion at Central Synapses , 1997, Neuron.

[22]  Leslie G. Ungerleider,et al.  Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque , 1990, The Journal of comparative neurology.

[23]  P. Dayan Fast oscillations in cortical circuits , 2000 .

[24]  Geoffrey E. Hinton Preface to the Special Issue on Connectionist Symbol Processing , 1990 .

[25]  Deliang Wang,et al.  Global competition and local cooperation in a network of neural oscillators , 1995 .

[26]  Robert M. French,et al.  A Neurobiologically Inspired Model of Working Memory Based on Neuronal Synchrony and Rythmicity , 1997, NCPW.

[27]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[28]  Leslie G. Valiant,et al.  Circuits of the mind , 1994 .

[29]  D. R. Mani Lokendra Shastri A Connectionist Solution to the Multiple Instantiation Problem using Temporal Synchrony , 1992 .

[30]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[31]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[32]  Dan Hammerstrom,et al.  Platform performance comparison of PALM network on Pentium 4 and FPGA , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[33]  T. Poggio,et al.  Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[34]  H B Barlow,et al.  Single units and sensation: a neuron doctrine for perceptual psychology? , 1972, Perception.

[35]  Ch. von der Malsburg,et al.  A neural cocktail-party processor , 1986, Biological Cybernetics.

[36]  Jerome A. Feldman,et al.  Dynamic connections in neural networks , 1990, Biological Cybernetics.

[37]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[38]  Mark Alan Fanty Learning in structured connectionist networks , 1988 .

[39]  F. Crick Function of the thalamic reticular complex: the searchlight hypothesis. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[41]  John A. Bullinaria,et al.  Analyzing the Internal Representations of Trained Neural Networks , 1997 .

[42]  Mikael Bodén,et al.  Representing Structure and Structured Representations in Connectionist Networks , 2019, Neural Network Perspectives on Cognition and Adaptive Robotics.

[43]  P. Milner A model for visual shape recognition. , 1974, Psychological review.

[44]  John McCarthy,et al.  Circumscription - A Form of Non-Monotonic Reasoning , 1980, Artif. Intell..

[45]  William B. Levy,et al.  Setting the Activity Level in Sparse Random Networks , 1994, Neural Computation.

[46]  F. Sommer,et al.  Neural Associative Memories , 1993 .

[47]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[48]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[49]  Lokendra Shastri,et al.  Biological Grounding of Recruitment Learning and Vicinal Algorithms in Long-Term Potentiation , 2001, Emergent Neural Computational Architectures Based on Neuroscience.

[50]  Wolf Singer,et al.  Time as Coding Space in Neocortical Processing: A Hypothesis , 1994 .

[51]  Jean Bullier,et al.  The Timing of Information Transfer in the Visual System , 1997 .

[52]  Scott E. Fahlman,et al.  The hashnet interconnection scheme , 1980 .

[53]  Lokendra Shastri,et al.  A Computational Model of Tractable Reasoning - Taking Inspiration from Cognition , 1993, IJCAI.

[54]  M. Abeles,et al.  Firing Rates and Weil-Timed Events in the Cerebral Cortex , 1994 .

[55]  T. Sejnowski,et al.  Synchronous oscillatory activity in sensory systems: new vistas on mechanisms , 1997, Current Opinion in Neurobiology.

[56]  Leslie G. Valiant,et al.  A neuroidal architecture for cognitive computation , 1998, ICALP.

[57]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[58]  Alexandros V. Gerbessiotis Random Graphs In A Neural Computation Model , 2003, Int. J. Comput. Math..

[59]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[60]  T. Sejnowski,et al.  Reliability of spike timing in neocortical neurons. , 1995, Science.

[61]  D. R. Mani,et al.  Massively Parallel Real-time Reasoning with Very Large Knowledge Bases: an Interim Report , 1994 .

[62]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[63]  Joachim Diederich,et al.  Instruction and High-level Learning in Connectionist Networks , 1989 .

[64]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[65]  Albert H. Titus,et al.  Toward an analog VLSI implementation of adaptive resonance theory (ART2) , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[66]  William B. Levy,et al.  The dynamics of sparse random networks , 1993, Biological Cybernetics.

[67]  P. König,et al.  Correlated firing in sensory-motor systems , 1995, Current Opinion in Neurobiology.

[68]  Wolfgang Maass,et al.  A Model for Fast Analog Computation Based on Unreliable Synapses , 2000, Neural Computation.

[69]  GrossbergS. Adaptive pattern classification and universal recoding , 1976 .

[70]  Jacques Sougné,et al.  Connectionism and the problem of multiple instantiation , 1998, Trends in Cognitive Sciences.

[71]  M. Page,et al.  Connectionist modelling in psychology: A localist manifesto , 2000, Behavioral and Brain Sciences.

[72]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[73]  Andreas K. Engel,et al.  Temporal Binding, Binocular Rivalry, and Consciousness , 1999, Consciousness and Cognition.

[74]  J. Horty Nonmonotonic Logic , 2001 .

[75]  Wolfgang Maass,et al.  Spiking Neurons , 1998, NC.

[76]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[77]  Dan Roth,et al.  SNoW User Guide , 1999 .

[78]  Leslie G. Valiant,et al.  Functionality in neural nets , 1988, COLT '88.

[79]  Stéphane Badel,et al.  A VLSI Hamming artificial neural network with k-winner-take-all and k-loser-take-all capability , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[80]  Istvan S. N. Berkeley,et al.  Making a middling mousetrap , 1993, Behavioral and Brain Sciences.

[81]  R. O’Reilly,et al.  Three forms of binding and their neural substrates: Alternatives to temporal synchrony , 2003 .

[82]  Lokendra Shastri,et al.  Advances in SHRUTI—A Neurally Motivated Model of Relational Knowledge Representation and Rapid Inference Using Temporal Synchrony , 1999, Applied Intelligence.

[83]  Cengiz Günay,et al.  Temporal binding as an inducer for connectionist recruitment learning over delayed lines , 2003, Neural Networks.