Computing with large random patterns

We describe a style of computing that differs from traditional numeric and symbolic computing and is suited for modeling neural networks. We focus on one aspect of ``neurocomputing,'' namely, computing with large random patterns, or high-dimensional random vectors, and ask what kind of computing they perform and whether they can help us understand how the brain processes information and how the mind works. Rapidly developing hardware technology will soon be able to produce the massive circuits that this style of computing requires. This chapter develops a theory on which the computing could be based.

[1]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[2]  Dmitri A. Rachkovskij,et al.  Binding and Normalization of Binary Sparse Distributed Representations by Context-Dependent Thinning , 2001, Neural Computation.

[3]  Geoffrey E. Hinton,et al.  Distributed representations and nested compositional structure , 1994 .

[4]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[5]  Arthur B. Markman,et al.  Analogy-- Watershed or Waterloo? Structural alignment and the development of connectionist models of analogy , 1992, NIPS 1992.

[6]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[7]  Hinrich Schütze,et al.  A Cooccurrence-Based Thesaurus and Two Applications to Information Retrieval , 1994, Inf. Process. Manag..

[8]  David J. Chalmers,et al.  Syntactic Transformations on Distributed Representations , 1990 .

[9]  Roger Wales,et al.  Connections, Binding, Unification and Analogical Promiscuity , 1998 .

[10]  Curt Burgess,et al.  The Dynamics of Meaning in Memory , 1998 .

[11]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[12]  Chris Eliasmith,et al.  Integrating structure and meaning: a distributed model of analogical mapping , 2001, Cogn. Sci..

[13]  Pentti Kanerva,et al.  Sparse distributed memory and related models , 1993 .

[14]  Mikael Bodén,et al.  Features of distributed representations for tree-structures: A study of RAAM , 1995 .

[15]  Pentti Kanerva,et al.  Binary Spatter-Coding of Ordered K-Tuples , 1996, ICANN.

[16]  G. Sjodin.  The Sparchunk code: a method to build higher-level structures in a sparsely encoded SDM , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[17]  Anders Holst,et al.  Random indexing of text samples for latent semantic analysis , 2000 .

[18]  Susan T. Dumais,et al.  Using latent semantic analysis to improve information retrieval , 1988, CHI 1988.

[19]  H. Schütze,et al.  Dimensions of meaning , 1992, Supercomputing '92.

[20]  Peter W. Foltz,et al.  Learning Human-like Knowledge by Singular Value Decomposition: A Progress Report , 1997, NIPS.

[21]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[22]  Santosh S. Vempala,et al.  Latent semantic indexing: a probabilistic analysis , 1998, PODS '98.

[23]  Louis A. Jaeckel.  An alternative design for a sparse distributed memory , 1989 .

[24]  D. Hofstadter.  Metamagical Themas: Questing for the Essence of Mind and Pattern , 1985 .

[25]  Samuel Kaski,et al.  Dimensionality reduction by random mapping: fast similarity computation for clustering , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[26]  R. Durrett.  Probability: Theory and Examples , 1993 .

[27]  Gunnar Sjödin.  Improving the Capacity of SDM , 1995 .

[28]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[29]  D. Gentner,et al.  The analogical mind : perspectives from cognitive science , 2001 .

[30]  Zellig S. Harris,et al.  Mathematical structures of language , 1968, Interscience tracts in pure and applied mathematics.

[31]  Melanie Mitchell,et al.  Analogy-making as perception - a computer model , 1993, Neural network modeling and connectionism.

[32]  Igor Aleksander,et al.  Introduction to Neural Computing , 1990 .

[33]  Bob Rehder,et al.  How Well Can Passage Meaning be Derived without Using Word Order? A Comparison of Latent Semantic Analysis and Humans , 1997 .

[34]  Gunnar Sjödin.  Getting More Information out of SDM , 1996, ICANN.

[35]  D. Gentner.  Structure‐Mapping: A Theoretical Framework for Analogy* , 1983 .

[36]  Douglas Hofstadter,et al.  The Copycat Project: An Experiment in Nondeterminism and Creative Analogies , 1984 .

[37]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[38]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[39]  Louis A. Jaeckel.  A class of designs for a sparse distributed memory , 1989 .

[40]  Dekang Lin,et al.  Using Syntactic Dependency as Local Context to Resolve Word Sense Ambiguity , 1997, ACL.

[41]  Jan Kristoferson.  Some results on activation and scaling of sparse distributed memory , 1998, Proceedings 5th Brazilian Symposium on Neural Networks (Cat. No.98EX209).

[42]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[43]  Jan Kristoferson.  Best Probability of Activation and Performance Comparisons for Several Designs of Sparse Distributed Memory , 1995 .