Putting distributed representations into context

ABSTRACT The merits of distributed representations are widely discussed but, I believe, largely misunderstood. My purpose in this paper is to put the issue of distributed representations into context. I will argue that the term distributed is only meaningful with respect to that which is being represented and that degree to which a localist or a distributed code is more desirable depends on the goals of the computation to be performed. I will also argue that localist codes play an essential role in symbolic neural computation.

[1]  John E. Hummel.  Localism as a first step toward symbolic representation , 2000 .

[2]  Minami Ito,et al.  Columns for visual features of objects in monkey inferotemporal cortex , 1992, Nature.

[3]  Simon J. Thorpe,et al.  Grandmother cells, Neocortical Dark Matter and very long term visual memories , 2011 .

[4]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  M. Page,et al.  Connectionist modelling in psychology: A localist manifesto , 2000, Behavioral and Brain Sciences.

[7]  R. Desimone,et al.  Visual properties of neurons in area V4 of the macaque: sensitivity to stimulus form. , 1987, Journal of neurophysiology.

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[10]  Brian Falkenhainer,et al.  The Structure-Mapping Engine: Algorithm and Examples , 1989, Artif. Intell..

[11]  Leonidas A A Doumas,et al.  A theory of the discovery and predication of relational concepts. , 2008, Psychological review.

[12]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[13]  John E. Hummel,et al.  Distributed representations of structure: A theory of analogical access and mapping. , 1997 .

[14]  Selmer Bringsjord,et al.  Analogy, explanation, and proof , 2014, Front. Hum. Neurosci..

[15]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[16]  Markus F. Damian,et al.  Neural networks learn highly selective representations in order to overcome the superposition catastrophe. , 2014, Psychological review.

[17]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[18]  J. Hegdé,et al.  Selectivity for Complex Shapes in Primate Visual Area V2 , 2000, The Journal of Neuroscience.

[19]  O. Andreassen,et al.  Mice Deficient in Cellular Glutathione Peroxidase Show Increased Vulnerability to Malonate, 3-Nitropropionic Acid, and 1-Methyl-4-Phenyl-1,2,5,6-Tetrahydropyridine , 2000, The Journal of Neuroscience.

[20]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[21]  James L. McClelland,et al.  Locating object knowledge in the brain: comment on Bowers's (2009) attempt to revive the grandmother cell hypothesis. , 2010, Psychological review.

[22]  W A Wickelgren,et al.  Webs, cell assemblies, and chunking in neural nets: introduction. , 1999, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[23]  Dedre Gentner,et al.  Structure-Mapping: A Theoretical Framework for Analogy , 1983, Cogn. Sci..

[24]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[25]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[26]  K. Holyoak,et al.  A symbolic-connectionist theory of relational inference and generalization. , 2003, Psychological review.

[27]  Paul Thagard,et al.  Analogical Mapping by Constraint Satisfaction , 1989, Cogn. Sci..

[28]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .