Using temporal binding for hierarchical recruitment of conjunctive concepts over delayed lines

The temporal correlation hypothesis proposes using distributed synchrony for the binding of different stimulus features. However, synchronized spikes must travel over cortical circuits that have varying-length pathways, leading to mismatched arrival times. This raises the question of how initial stimulus-dependent synchrony might be preserved at a destination binding site. Earlier, we proposed constraints on tolerance and segregation parameters for a phase-coding approach, within cortical circuits, to address this question [C. Gunay, A.S. Maida, Temporal binding as an inducer for connectionist recruitment learning over delayed lines, Neural Networks 16 (5-6) (2003) 593-600]. The purpose of the present paper is twofold. First, we conduct simulation studies that explore the effectiveness of the proposed constraints. Second, we place the studies in a broader context of synchrony-driven recruitment learning [L. Shastri, V. Ajjanagadde, From simple associations to systematic reasoning: a connectionist representation of rules, variables, and dynamic bindings using temporal synchrony, Behav. Brain Sci. 16 (3) (1993) 417-451; L.G. Valiant, Circuits of the Mind, Oxford University Press, Oxford, 1994] which brings together von der Malsburg's temporal binding [C. von der Malsburg, The correlation theory of brain function, in: E. Domany, J.L. van Hemmen, K. Schulten (Ed.), Models of Neural Networks, vol. 2, Physics of Neural Networks, Chapter 2, Springer, New York, 1994, pp. 95-120, (Originally appeared as a Technical Report at the Max-Planck Institute for Biophysical Chemistry, Gottingen, 1981)] and Feldman's recruitment learning [J.A. Feldman, Dynamic connections in neural networks, Biol. Cybern. 46 (1982) 27-39]. A network based on Valiant's neuroidal architecture is used to implement synchrony-driven recruitment learning. Complementing similar approaches, we use a continuous-time learning procedure allowing computation with spiking neurons. The viability of the proposed binding scheme is investigated by conducting simulation studies which examine binding errors. In the simulation, binding errors cause the formation of illusory conjunctions among features belonging to separate objects. Our results indicate that when tolerance and segregation parameters obey our proposed constraints, the sets of correct bindings are dominant over sets of spurious bindings in reasonable operating conditions. We also improve the stability of the recruitment method in deep hierarchies for use in limited size structures suitable for computer simulations. e also improve the stability of the recruitment method in deep hierarchies for use in limited size structures suitable for computer simulations.

[1]  Lokendra Shastri,et al.  A computationally efficient abstraction of long-term potentiation , 2002, Neurocomputing.

[2]  O Jensen,et al.  Novel lists of 7 +/- 2 known items can be reliably stored in an oscillatory short-term memory network: interaction with long-term memory. , 1996, Learning & memory.

[3]  Lokendra Shastri,et al.  Recruitment of binding and binding-error detector circuits via long-term potentiation , 1999, Neurocomputing.

[4]  Ch. von der Malsburg,et al.  A neural cocktail-party processor , 1986, Biological Cybernetics.

[5]  Jerome A. Feldman,et al.  Dynamic connections in neural networks , 1990, Biological Cybernetics.

[6]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[7]  Mark Alan Fanty Learning in structured connectionist networks , 1988 .

[8]  Lokendra Shastri,et al.  Types and Quantifiers in SHRUTI: A Connectionist Model of Rapid Reasoning and Relational Processing , 1998, Hybrid Neural Systems.

[9]  Guy L. Steele,et al.  Java Language Specification, Second Edition: The Java Series , 2000 .

[10]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[11]  J J Hopfield,et al.  Rapid local synchronization of action potentials: toward computation with coupled integrate-and-fire neurons. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[12]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[13]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[14]  Jacques Sougné,et al.  Period doubling as a means of representing multiply instantiated entities , 1998 .

[15]  Leslie G. Ungerleider,et al.  Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque , 1990, The Journal of comparative neurology.

[16]  A. Treisman The binding problem , 1996, Current Opinion in Neurobiology.

[17]  Andreas K. Engel,et al.  Temporal Binding, Binocular Rivalry, and Consciousness , 1999, Consciousness and Cognition.

[18]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[19]  W A Wickelgren,et al.  Chunking and consolidation: a theoretical synthesis of semantic networks, configuring in conditioning, S--R versus congenitive learning, normal forgetting, the amnesic syndrome, and the hippocampal arousal system. , 1979, Psychological review.

[20]  Daniel Lehmann,et al.  Modeling Compositionality by Dynamic Binding of Synfire Chains , 2004, Journal of Computational Neuroscience.

[21]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[22]  H. W. Veen,et al.  Handbook of Biological Physics , 1996 .

[23]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[24]  P. König,et al.  Correlated firing in sensory-motor systems , 1995, Current Opinion in Neurobiology.

[25]  Alexandros V. Gerbessiotis,et al.  Topics in parallel and distributed computation , 1993 .

[26]  C. Koch,et al.  A brief history of time (constants). , 1996, Cerebral cortex.

[27]  Lokendra Shastri,et al.  Semantic Networks: An Evidential Formalization and Its Connectionist Realization , 1988 .

[28]  D. B. Davis,et al.  Sun Microsystems Inc. , 1993 .

[29]  Cengiz Günay,et al.  A stochastic population approach to the problem of stable recruitment hierarchies in spiking neural networks , 2005, Biological Cybernetics.

[30]  Jim Austin,et al.  Emergent Neural Computational Architectures Based on Neuroscience: Towards Neuroscience-Inspired Computing , 2001 .

[31]  E. Bizzi,et al.  The Cognitive Neurosciences , 1996 .

[32]  Axel Cleeremans,et al.  The Unity of Consciousness: Binding, Integration, and Dissociation , 2003 .

[33]  C. Koch,et al.  Is perception discrete or continuous? , 2003, Trends in Cognitive Sciences.

[34]  Wayne A. Wickelgren,et al.  Chunking and consolidation: A theoretical synthesis of semantic networks configuring in conditioning , 1979 .

[35]  Wolfgang Maass,et al.  Spiking Neurons , 1998, NC.

[36]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[37]  William J. Bowhill,et al.  Design of High-Performance Microprocessor Circuits , 2001 .

[38]  M. Abeles,et al.  Firing Rates and Weil-Timed Events in the Cerebral Cortex , 1994 .

[39]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[40]  T. Sejnowski,et al.  Synchronous oscillatory activity in sensory systems: new vistas on mechanisms , 1997, Current Opinion in Neurobiology.

[41]  C. Gunay,et al.  The required measures of phase segregation in distributed cortical processing , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[42]  B. McNaughton,et al.  Hippocampal synaptic enhancement and information storage within a distributed memory system , 1987, Trends in Neurosciences.

[43]  Leslie G. Valiant,et al.  A neuroidal architecture for cognitive computation , 1998, ICALP.

[44]  Alexandros V. Gerbessiotis Random Graphs In A Neural Computation Model , 2003, Int. J. Comput. Math..

[45]  Nick Littlestone,et al.  Redundant noisy attributes, attribute errors, and linear-threshold learning using winnow , 1991, COLT '91.

[46]  Martin Schneider,et al.  Activity-Dependent Development of Axonal and Dendritic Delays, or, Why Synaptic Transmission Should Be Unreliable , 2002, Neural Computation.

[47]  C. Gunay,et al.  Using temporal binding for connectionist recruitment learning over delayed lines , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[48]  Cengiz Günay,et al.  Using Temporal Binding for Robust Connectionist Recruitment Learning over Delayed Lines ∗ , 2003 .

[49]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[50]  Benjamin Kuipers,et al.  Qualitative reasoning: Modeling and simulation with incomplete knowledge , 1994, Autom..

[51]  C. Gunay,et al.  Simulation of planar I/F networks with delayed connections , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[52]  R. Shiffrin,et al.  Controlled and automatic human information processing: I , 1977 .

[53]  Alfred Nischwitz,et al.  Local lateral inhibition: a key to spike synchronization? , 1995, Biological Cybernetics.

[54]  Leslie G. Valiant,et al.  Functionality in neural nets , 1988, COLT '88.

[55]  W. Gerstner,et al.  Chapter 12 A framework for spiking neuron models: The spike response model , 2001 .

[56]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[57]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[58]  Stefan Wermter,et al.  Hybrid Neural Systems, revised papers from a workshop , 1998 .

[59]  Cengiz Günay,et al.  Hierarchical learning of conjunctive concepts in spiking neural networks , 2003 .

[60]  Jerome A. Feldman,et al.  Computational constraints on higher neural representations , 1993 .

[61]  Peter König,et al.  Binding by temporal structure in multiple feature domains of an oscillatory neuronal network , 1994, Biological Cybernetics.

[62]  G. Ellis‐Davies,et al.  Structural basis of long-term potentiation in single dendritic spines , 2004, Nature.

[63]  Jacques Sougné,et al.  Connectionism and the problem of multiple instantiation , 1998, Trends in Cognitive Sciences.

[64]  R. O’Reilly,et al.  Three forms of binding and their neural substrates: Alternatives to temporal synchrony , 2003 .

[65]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[66]  Lokendra Shastri,et al.  A computational model of episodic memory formation in the hippocampal system , 2001, Neurocomputing.

[67]  Lokendra Shastri,et al.  Advances in SHRUTI—A Neurally Motivated Model of Relational Knowledge Representation and Rapid Inference Using Temporal Synchrony , 1999, Applied Intelligence.

[68]  Cengiz Günay,et al.  Temporal binding as an inducer for connectionist recruitment learning over delayed lines , 2003, Neural Networks.

[69]  N. Kasthuri,et al.  Long-term dendritic spine stability in the adult cortex , 2002, Nature.

[70]  Günther Palm,et al.  Spiking Associative Memory and Scene Segmentation by Synchronization of Cortical Activity , 2001, Emergent Neural Computational Architectures Based on Neuroscience.

[71]  Lokendra Shastri,et al.  A Computational Model of Tractable Reasoning - Taking Inspiration from Cognition , 1993, IJCAI.

[72]  David Bailey,et al.  Layered Hybrid Connectionist Models for Cognitive Science , 1998, Hybrid Neural Systems.

[73]  C. Malsburg Binding in models of perception and brain function , 1995, Current Opinion in Neurobiology.

[74]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[75]  Lokendra Shastri,et al.  Biological Grounding of Recruitment Learning and Vicinal Algorithms in Long-Term Potentiation , 2001, Emergent Neural Computational Architectures Based on Neuroscience.

[76]  Wolf Singer,et al.  Time as Coding Space in Neocortical Processing: A Hypothesis , 1994 .

[77]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[78]  Jean Bullier,et al.  The Timing of Information Transfer in the Visual System , 1997 .

[79]  DeLiang Wang,et al.  Synchrony and Desynchrony in Integrate-and-Fire Oscillators , 1999, Neural Computation.

[80]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[81]  Lokendra Shastri,et al.  A Model of Rapid Memory Formation in the Hippocampal System , 2007 .

[82]  Antony Browne,et al.  Connectionist inference models , 2001, Neural Networks.

[83]  D. Perrett,et al.  Visual neurones responsive to faces in the monkey temporal cortex , 2004, Experimental Brain Research.

[84]  L. Shastri Episodic memory and cortico–hippocampal interactions , 2002, Trends in Cognitive Sciences.

[85]  Leslie G. Valiant,et al.  Circuits of the mind , 1994 .

[86]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[87]  J. Davenport Editor , 1960 .

[88]  Deliang Wang,et al.  Global competition and local cooperation in a network of neural oscillators , 1995 .

[89]  Robert M. French,et al.  A Neurobiologically Inspired Model of Working Memory Based on Neuronal Synchrony and Rythmicity , 1997, NCPW.

[90]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[91]  J E Lisman,et al.  Storage of 7 +/- 2 short-term memories in oscillatory subcycles , 1995, Science.

[92]  S. Strogatz,et al.  Synchronization of pulse-coupled biological oscillators , 1990 .

[93]  D. R. Mani Lokendra Shastri A Connectionist Solution to the Multiple Instantiation Problem using Temporal Synchrony , 1992 .

[94]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .