Getting symbols out of a neural architecture

Traditional connectionist networks are sharply limited as general accounts of human perception and cognition because they are unable to represent relational ideas such as loves (John, Mary) or bigger-than (Volkswagen, breadbox) in a way that allows them to be manipulated as explicitly relational structures. This paper reviews and critiques the four major responses to this problem in the modelling community: (1) reject connectionism (in any form) in favour of traditional symbolic approaches to modelling the mind; (2) reject the idea that mental representations are symbolic (i.e. reject the idea that we can represent relations); and (3) attempt to represent symbolic structures in a connectionist/neural architecture by finding a way to represent role–filler bindings. Approach (3) is further subdivided into (3a) approaches based on varieties of conjunctive coding and (3b) approaches based on dynamic role–filler binding. I will argue that (3b) is necessary to get symbolic processing out of a neural computing architecture. Specifically, I will argue that vector addition is both the best way to accomplish dynamic binding and an essential part of the proper treatment of symbols in a neural architecture.

[1]  K. Holyoak,et al.  A System for Relational Reasoning in Human Prefrontal Cortex , 1999 .

[2]  C..  Utilizing Time : Asynchronous Binding , 1998 .

[3]  Simon D. Levy,et al.  A distributed basis for analogical mapping , 2009 .

[4]  Noam Chomsky.  Review of B.F. Skinner, Verbal Behavior , 1959 .

[5]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[6]  J. Hummel,et al.  The role of attention in priming for left-right reflections of object images: evidence for a dual representation of object shape. , 1998, Journal of experimental psychology. Human perception and performance.

[7]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[8]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[9]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[10]  Markus F. Damian,et al.  A fundamental limitation of the conjunctive codes learned in PDP models of cognition: comment on Botvinick and Plaut (2006). , 2009, Psychological review.

[11]  Irving Biederman,et al.  Object recognition, attention, and action , 2007 .

[12]  John E. Hummel,et al.  Distributed representations of structure: A theory of analogical access and mapping. , 1997 .

[13]  K. Holyoak,et al.  The Cambridge handbook of thinking and reasoning , 2005 .

[14]  Robert L. Goldstone,et al.  Relational similarity and the nonindependence of features in similarity judgments , 1991, Cognitive Psychology.

[15]  J. E. Hummel,et al.  No way to start a space program. Commentary on R. Leech, D. Mareschal and RP Cooper, Analogy as relational priming: A developmental and computational perspective on the origins of a complex skill , 2008 .

[16]  Leonidas A A Doumas,et al.  A theory of the discovery and predication of relational concepts. , 2008, Psychological review.

[17]  Stella Vosniadou,et al.  Similarity and analogical reasoning: Similarity and Analogical Reasoning , 1989 .

[18]  John E. Hummel,et al.  No way to start a space program: Associationism as a launch pad for analogical reasoning , 2008, Behavioral and Brain Sciences.

[19]  J. Hummel.  Complementary solutions to the binding problem in vision: Implications for shape perception and object recognition , 2001 .

[20]  J. Hummel,et al.  Modeling Human Mental Representations: What Works, What doesn't and Why? , 2005 .

[21]  Linda B. Smith.  From global similarities to kinds of similarities: the construction of dimensions in development , 1989 .

[22]  D. Mareschal,et al.  Analogy as relational priming: The challenge of self-reflection , 2008, Behavioral and Brain Sciences.

[23]  Bradley C. Love,et al.  Utilizing lime: Asynchronous Binding , 1998, NIPS.

[24]  Zenon W. Pylyshyn,et al.  Connectionism and cognitive architecture , 1993 .

[25]  Ross W. Gayler.  Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience , 2004, ArXiv.

[26]  Jeremy R. Reynolds,et al.  Developing PFC representations using reinforcement learning , 2009, Cognition.

[27]  Leonidas A. A. Doumas,et al.  A Computational Account of the Development of the Generalization of Shape Information , 2010, Cogn. Sci..

[28]  Daniel C. Krawczyk,et al.  A Neurocomputational Model of Analogical Reasoning and its Breakdown in Frontotemporal Lobar Degeneration , 2004, Journal of Cognitive Neuroscience.

[29]  S E Palmer,et al.  Structural aspects of visual similarity , 1978, Memory & cognition.

[30]  Volker Thoma,et al.  Object Recognition: Attention and Dual Routes , 2007 .

[31]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[32]  Susan Bowsfield.  The Symbolic Species: The Co-Evolution of Language and the Brain , 2004 .

[33]  Derek C. Penn,et al.  Darwin's mistake: Explaining the discontinuity between human and nonhuman minds , 2008, Behavioral and Brain Sciences.

[34]  G. Marcus.  Rethinking Eliminative Connectionism , 1998, Cognitive Psychology.

[35]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[36]  K. Holyoak,et al.  Mental Leaps: Analogy in Creative Thought , 1994 .

[37]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[38]  John E. Hummel,et al.  Symbolic Versus Associative Learning , 2010, Cogn. Sci..

[39]  John E. Hummel,et al.  Priming of plane-rotated objects depends on attention and view familiarity , 2006 .

[40]  John E. Hummel,et al.  Automatic priming for translation- and scale-invariant representations of object shape , 2002 .

[41]  N. Cowan.  The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[42]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[43]  Irving Biederman,et al.  Comprar Object Recognition, Attention, and Action | Osaka, Naoyuki | 9784431730187 | Springer , 2007 .

[44]  J. Desmedt,et al.  Transient phase-locking of 40 Hz electrical oscillations in prefrontal and parietal human cortex reflects the process of conscious somatic perception , 1994, Neuroscience Letters.

[45]  J. Fuster.  Frontal lobes , 1993, Current Opinion in Neurobiology.

[46]  James L. McClelland,et al.  Précis of Semantic Cognition: A Parallel Distributed Processing Approach , 2008, Behavioral and Brain Sciences.

[47]  J. Hummel,et al.  Evidence for holistic representations of ignored images and analytic representations of attended images. , 2004, Journal of experimental psychology. Human perception and performance.

[48]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[49]  Donald Favareau.  The Symbolic Species: The Co-evolution of Language and the Brain , 1998 .

[50]  E. Bizzi,et al.  The Cognitive Neurosciences , 1996 .

[51]  E. Miller,et al.  Neural Activity in the Primate Prefrontal Cortex during Associative Learning , 1998, Neuron.

[52]  Jeffrey S Bowers,et al.  On the biological plausibility of grandmother cells: implications for neural network theories in psychology and neuroscience. , 2009, Psychological review.

[53]  Vladimir M. Sloutsky,et al.  Analogy is to priming as relations are to transformations , 2008, Behavioral and Brain Sciences.

[54]  Robert G. Morrison,et al.  Neurocognitive process constraints on analogy: What changes to allow children to reason like adults? , 2008, Behavioral and Brain Sciences.

[55]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[56]  Alexander A. Petrov,et al.  Relational priming plays a supporting but not leading role in adult analogy-making , 2008, Behavioral and Brain Sciences.

[57]  C. Peirce,et al.  Philosophical Writings of Peirce , 1955 .

[58]  R. O’Reilly,et al.  Three forms of binding and their neural substrates: Alternatives to temporal synchrony , 2003 .

[59]  Dedre Gentner,et al.  Structure-Mapping: A Theoretical Framework for Analogy , 1983, Cogn. Sci..

[60]  D. Gentner,et al.  The analogical mind : perspectives from cognitive science , 2001 .

[61]  John E. Hummel,et al.  Finding similarity in a model of relational reasoning , 2009, Cognitive Systems Research.

[62]  Tony Plate,et al.  Holographic Reduced Representations: Convolution Algebra for Compositional Distributed Representations , 1991, IJCAI.

[63]  Indre V. Viskontas,et al.  Relational integration, inhibition, and analogical reasoning in older adults. , 2004, Psychology and aging.

[64]  G. Halford.  Analogical Reasoning and Conceptual Complexity in Cognitive Development , 1992 .

[65]  K. Holyoak,et al.  A symbolic-connectionist theory of relational inference and generalization. , 2003, Psychological review.

[66]  Frederick A. Zehrer,et al.  Review of Verbal behavior. , 1959 .

[67]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[68]  Leonidas A. A. Doumas,et al.  Developing structured representations , 2008 .

[69]  Robert M. French.  A new manifesto for child development research , 2008 .