Representation and Processing of Structures with Binary Sparse Distributed Codes

The schemes for compositional distributed representations include those allowing on-the-fly construction of fixed dimensionality codevectors to encode structures of various complexity. Similarity of such codevectors takes into account both structural and semantic similarity of represented structures. We provide a comparative description of sparse binary distributed representation developed in the framework of the associative-projective neural network architecture and the more well known holographic reduced representations of T.A. Plate (1995) and binary spatter codes of P. Kanerva (1996). The key procedure in associative-projective neural networks is context-dependent thinning which binds codevectors and maintains their sparseness. The codevectors are stored in structured memory array which can be realized as distributed auto-associative memory. Examples of distributed representation of structured data are given. Fast estimation of the similarity of analogical episodes by the overlap of their codevectors is used in the modeling of analogical reasoning both for retrieval of analogs from memory and for analogical mapping.

[1]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[2]  J. J. Hopfield,et al.  ‘Unlearning’ has a stabilizing effect in collective memories , 1983, Nature.

[3]  Paul Thagard,et al.  Integrating structure and meaning: a distributed model of analogical mapping , 2001 .

[4]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[5]  D. Gentner.  Structure‐Mapping: A Theoretical Framework for Analogy* , 1983 .

[6]  Alexander A. Frolov,et al.  Informational Capacity and Recall Quality in Sparsely Encoded Hopfield-like Neural Network: Analytical Approaches and Computer Simulation , 1997, Neural Networks.

[7]  C. von der Malsburg,et al.  Am I Thinking Assemblies , 1986 .

[8]  Örjan Ekeberg,et al.  Reliability and Speed of Recall in an Associative Network , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  D. Gentner,et al.  Structure mapping in analogy and similarity. , 1997 .

[10]  P. Milner.  A model for visual shape recognition. , 1974, Psychological review.

[11]  T. E. Lange,et al.  Below the Surface: Analogical Similarity and Retrieval Competition in Reminding , 1994, Cognitive Psychology.

[12]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[13]  Kenneth D. Forbus,et al.  MAC/FAC: A Model of Similarity-Based Retrieval , 1995, Cogn. Sci..

[14]  S. Phillips,et al.  Processing capacity defined by relational complexity: implications for comparative, developmental, and cognitive psychology. , 1998, The Behavioral and brain sciences.

[15]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[16]  Paul Thagard,et al.  Analogical Mapping by Constraint Satisfaction , 1989, Cogn. Sci..

[17]  M. Tsodyks.  ASSOCIATIVE MEMORY IN NEURAL NETWORKS WITH THE HEBBIAN LEARNING RULE , 1989 .

[18]  Alessandro Sperduti,et al.  A general framework for adaptive processing of data structures , 1998, IEEE Trans. Neural Networks.

[19]  Shun-ichi Amari,et al.  Characteristics of sparsely encoded associative memory , 1989, Neural Networks.

[20]  Björn Levin,et al.  Holistic higher-level structure-forming algorithms , 1998 .

[21]  Arthur B. Markman,et al.  Analogy-based reasoning , 1998 .

[22]  Dmitri A. Rachkovskij,et al.  Binding and Normalization of Binary Sparse Distributed Representations by Context-Dependent Thinning , 2001, Neural Computation.

[23]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[24]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[25]  Alessandro Sperduti,et al.  Modular Labeling RAAM , 1995, ICANNGA.

[26]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[27]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[28]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[29]  B. Ross.  Distinguishing Types of Superficial Similarities: Different Effects on the Access and Use of Earlier Problems , 1989 .

[30]  John E. Hummel,et al.  Distributed representations of structure: A theory of analogical access and mapping. , 1997 .

[31]  Alessandro Sperduti,et al.  Labelling Recursive Auto-associative Memory , 1994, Connect. Sci..

[32]  Arthur B. Markman,et al.  Structural Alignment in Similarity and its Influence on Category Structure , 1997 .

[33]  Jerome A. Feldman,et al.  Neural Representation of Conceptual Knowledge. , 1986 .

[34]  Brian Falkenhainer,et al.  The Structure-Mapping Engine: Algorithm and Examples , 1989, Artif. Intell..

[35]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[36]  Pentti Kanerva,et al.  Binary Spatter-Coding of Ordered K-Tuples , 1996, ICANN.

[37]  G. Sjodin.  The Sparchunk code: a method to build higher-level structures in a sparsely encoded SDM , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[38]  Tony A. Plate,et al.  Analogy retrieval and processing with distributed vector representations , 2000, Expert Syst. J. Knowl. Eng..

[39]  D. Gentner,et al.  Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences , 1997, Cognitive Psychology.

[40]  Paul Thagard,et al.  Analog Wetrieval by Constraint Satisfaction , 1990, Artif. Intell..

[41]  David S. Touretzky,et al.  BoltzCONS: Dynamic Symbol Structures in a Connectionist Network , 1990, Artif. Intell..