Binding and multiple instantiation in a distributed network of spiking nodes

An implementation of a distributed connectionist network of spiking neuron-like elements is presented. Spiking nodes fire at a precise moment and transmit their activation, with particular strengths and delays, to nodes connected to them. When the potential of the node reaches a particular threshold, it emits a spike. Thereafter, the potential is reset to a resting value. The receiving nodes accumulate potential, but also slowly lose their potential through decay. As with real neurons, after firing there is a short refractory period during which the node will be completely insensitive to incoming signals, after which its sensitivity will slowly increase. Precise timing properties are used to represent symbols in a distributed manner and to solve the problems of variable binding and multiple instantiation. Several predictions about human short-term memory, predicate processing, complex reasoning and multiple instantiation arise from this model. This network shows how symbolic processing can be achieved using neurobiologically and psychologically plausible mechanisms that also have the advantages of generalization and noise tolerance found in connectionist networks.

[1]  D. R. Mani,et al.  Reflexive Reasoning with Multiple Instantiation in a Connectionist Reasoning System with a Type Hierarchy , 1993 .

[2]  E. Vogel,et al.  Response from Luck and Vogel , 1998, Trends in Cognitive Sciences.

[3]  W. Singer,et al.  Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. , 1992, Science.

[4]  W. Singer,et al.  Neuronal assemblies: necessity, signature and detectability , 1997, Trends in Cognitive Sciences.

[5]  Dean A. Pomerleau,et al.  The meta-generalized delta rule : a new algorithm for learning in connectionist networks , 1987 .

[6]  Ron Sun,et al.  On Variable Binding in Connectionist Networks , 1992 .

[7]  Jacques Sougné,et al.  A Connectionist Model of Reflective Reasoning Using Temporal Properties of Node Firing , 1996 .

[8]  William A. MacKay,et al.  Synchronized neuronal oscillations and their role in motor processes , 1997, Trends in Cognitive Sciences.

[9]  Dedre Gentner,et al.  Systematicity and Surface Similarity in the Development of Analogy , 1986, Cogn. Sci..

[10]  John E. Hummel,et al.  Distributed representations of structure: A theory of analogical access and mapping. , 1997 .

[11]  J E Lisman,et al.  Storage of 7 +/- 2 short-term memories in oscillatory subcycles , 1995, Science.

[12]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[13]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[14]  Wolf Singer.  Synchronization of neuronal responses as a putative binding mechanism , 1998 .

[15]  T. Shallice,et al.  Please Scroll down for Article the Quarterly Journal of Experimental Psychology Independent Functioning of Verbal Memory Stores: a Neuropsychological Study Independent Functioning of Verbal Memory Stores : a Neuropsychological Study , 2022 .

[16]  Donald L. Fisher,et al.  Central capacity limits in consistent mapping, visual search tasks: Four channels or more? , 1984, Cognitive Psychology.

[17]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[18]  David Horn,et al.  The Importance of Noise for Segmentation and Binding in Dynamical Neural Systems , 1996, Int. J. Neural Syst..

[19]  Daphne N. Yu,et al.  High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. , 1997, Cerebral cortex.

[20]  N. Cowan.  Visual and auditory working memory capacity , 1998, Trends in Cognitive Sciences.

[21]  Robert M. French,et al.  Why localist connectionist models are inadequate for categorization , 2000, Behavioral and Brain Sciences.

[22]  N. Cowan.  The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[23]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[24]  James L. McClelland.  The programmable blackboard model of reading , 1986 .

[25]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[26]  R. Ivry,et al.  Effect of feature similarity on illusory conjunctions , 1991, Perception & psychophysics.

[27]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[28]  John McCarthy,et al.  Epistemological challenges for connectionism , 1988, Behavioral and Brain Sciences.

[29]  O Jensen,et al.  Novel lists of 7 +/- 2 known items can be reliably stored in an oscillatory short-term memory network: interaction with long-term memory. , 1996, Learning & memory.

[30]  W. Singer,et al.  In search of common foundations for cortical computation , 1997, Behavioral and Brain Sciences.

[31]  Graeme S. Halford,et al.  Does a concurrent memory load interfere with reasoning? , 1984 .

[32]  S. Grossberg,et al.  Psychological Review , 2003 .

[33]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[34]  Robert M. French,et al.  The Subtlety of Sameness: A Theory and Computer Model of Analogy-Making , 1995 .

[35]  E. Fetz,et al.  Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Douglas Hofstadter,et al.  The Copycat Project: An Experiment in Nondeterminism and Creative Analogies , 1984 .

[37]  H S Terrace,et al.  Chunking during serial learning by a pigeon: I. Basic evidence. , 1991, Journal of experimental psychology. Animal behavior processes.

[38]  P König,et al.  Direct physiological evidence for scene segmentation by temporal coding. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[39]  J. Winson.  Loss of hippocampal theta rhythm results in spatial memory deficit in the rat. , 1978, Science.

[40]  A. Baddeley,et al.  Amnesia and the distinction between long- and short-term memory. , 1970 .

[41]  S. Yantis.  Multielement visual tracking: Attention and perceptual organization , 1992, Cognitive Psychology.

[42]  Gordon M. Shepherd,et al.  Olfactory cortex , 1998 .

[43]  perspectiveVenkat AjjanagaddeWilhelm-Schickard Institute,et al.  Unclear Distinctions Lead to Unnecessary Shortcomings: Examining the Rule Vs Fact, Role Vs Ller, and Type Vs Predicate Distinctions from a Connectionist Representation and Reasoning Perspective , 1994 .

[44]  Wulfram Gerstner,et al.  Spiking neurons , 1999 .

[45]  Wolfgang Maass,et al.  On Computation with Pulses , 2000, Electron. Colloquium Comput. Complex..

[46]  J. Sougné.  Simulating conditional reasoning containing negations : A computer model and human data , 2000 .

[47]  A V Herz,et al.  Neural codes: firing rates and beyond. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Michael C. Mackey,et al.  From Clocks to Chaos , 1988 .

[49]  Jacques Sougné,et al.  A Learning Algorithm for Synfire Chains , 2000, NCPW.

[50]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[51]  Neil Burgess,et al.  Theta activity, virtual navigation and the human hippocampus , 1999, Trends in Cognitive Sciences.

[52]  G. Halford,et al.  A category theory approach to cognitive development , 1980, Cognitive Psychology.

[53]  Hatsuo Hayashi,et al.  Chaotic and phase-locked responses of the somatosensory cortex to a periodic medial lemniscus stimulation in the anesthetized rat , 1996, Brain Research.

[54]  Robert M. French,et al.  A Neurobiologically Inspired Model of Working Memory Based on Neuronal Synchrony and Rythmicity , 1997, NCPW.

[55]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[56]  W. Singer,et al.  The Role of Neuronal Synchronization in Response Selection: A Biologically Plausible Theory of Structured Representations in the Visual Cortex , 1996, Journal of Cognitive Neuroscience.

[57]  Jacques Sougné,et al.  Connectionism and the problem of multiple instantiation , 1998, Trends in Cognitive Sciences.

[58]  Jacques Sougné,et al.  Period doubling as a means of representing multiply instantiated entities , 1998 .

[59]  A Treisman,et al.  Feature analysis in early vision: evidence from search asymmetries. , 1988, Psychological review.

[60]  John A. Barnden,et al.  Encoding techniques for complex information structures in connectionist systems , 1991 .

[61]  J. Rinzel,et al.  Oscillatory and bursting properties of neurons , 1998 .

[62]  Michael G. Dyer,et al.  High-level Inferencing in a Connectionist Network , 1989 .

[63]  Frank H. Eeckman,et al.  Attentional Network Streams of Synchronized 40Hz Activity in a Cortical Architecture of Coupled Osci , 1997 .

[64]  N. Kanwisher.  Repetition blindness: Type recognition without token individuation , 1987, Cognition.

[65]  Venkat Ajjanagadde.  Unclear Distinctions Lead to Unnecessary Shortcomings: Examining the Rule versus Fact, Role versus Filler, and Type versus Predicate Distinctions from a Connectionist Representation and Reasoning Perspective , 1994, AAAI.

[66]  Geoffrey M. Ghose,et al.  Specialized Representations in Visual Cortex A Role for Binding? , 1999, Neuron.

[67]  J. Rinzel,et al.  The role of dendrites in auditory coincidence detection , 1998, Nature.

[68]  James Henderson,et al.  Description-based parsing in a connectionist network , 1995 .

[69]  Jacques Sougné,et al.  INFERNET : A neurocomputational model of binding and inference , 1999 .

[70]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[71]  Trent E. Lange.  Lexical and Pragmatic Disambiguation and Re-Interpretation in Connectionist Networks , 1992, Int. J. Man Mach. Stud..

[72]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[73]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[74]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[75]  Stefan Kasicki,et al.  Theta-like rhythm in depth EEG activity of hypothalamic areas during spontaneous or electrically induced locomotion in the rat , 1995, Brain Research.

[76]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[77]  David Horn,et al.  Segmentation and Binding in Oscillatory Neural Systems , 1995 .

[78]  W. Singer.  Synchronization of cortical activity and its putative role in information processing and learning. , 1993, Annual review of physiology.

[79]  Keith Stenning,et al.  Extension of the temporal synchrony approach to dynamic variable bindingin a connectionist inference system , 1995, Knowl. Based Syst..

[80]  J. Lisman,et al.  Physiologically realistic formation of autoassociative memory in networks with theta/gamma oscillations: role of fast NMDA channels. , 1996, Learning & memory.

[81]  Paul Antoine Salin,et al.  Spatial and temporal coherence in cortico-cortical connections: a cross-correlation study in areas 17 and 18 in the cat. , 1992, Visual neuroscience.

[82]  Morton B. Brown,et al.  372: The Anova and Multiple Comparisons for Data with Heterogeneous Variances , 1974 .

[83]  Jerome A. Feldman,et al.  Neural Representation of Conceptual Knowledge. , 1986 .

[84]  O Jensen,et al.  An Oscillatory Short-Term Memory Buffer Model Can Account for Data on the Sternberg Task , 1998, The Journal of Neuroscience.

[85]  James L. McClelland,et al.  Explorations in Parallel Distributed Processing - Macintosh version: A Handbook of Models, Programs, and Exercises , 1989 .

[86]  Joseph R. Madsen,et al.  Human theta oscillations exhibit task dependence during virtual maze navigation , 1999, Nature.

[87]  P König,et al.  Synchronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[88]  W. Singer,et al.  Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex , 1991, Science.

[89]  S. Grossberg,et al.  Cortical Synchronization and Perceptual Framing , 1997, Journal of Cognitive Neuroscience.

[90]  A. Mikami,et al.  Oscillatory neuronal activity related to visual short-term memory in monkey temporal pole. , 1992, Neuroreport.

[91]  J. Byrne,et al.  Routes to chaos in a model of a bursting neuron. , 1990, Biophysical journal.

[92]  W. Singer,et al.  Stimulus‐Dependent Neuronal Oscillations in Cat Visual Cortex: Inter‐Columnar Interaction as Determined by Cross‐Correlation Analysis , 1990, The European journal of neuroscience.