Holographic reduced representations

Associative memories are conventionally used to represent data with very simple structure: sets of pairs of vectors. This paper describes a method for representing more complex compositional structure in distributed representations. The method uses circular convolution to associate items, which are represented by vectors. Arbitrary variable bindings, short sequences of various lengths, simple frame-like structures, and reduced representations can be represented in a fixed width vector. These representations are items in their own right and can be used in constructing compositional structures. The noisy reconstructions extracted from convolution memories can be cleaned up by using a separate associative memory that has good reconstructive properties.

[1]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[2]  Alan R. Jones,et al.  Fast Fourier Transform , 1970, SIGP.

[3]  Richard A. Roberts,et al.  Signals and linear systems , 1973 .

[4]  Janet Metcalfe,et al.  A composite holographic associative recall model , 1982 .

[5]  J. Eich.  A composite holographic associative recall model. , 1982 .

[6]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[7]  B. Murdock.  A Theory for the Storage and Retrieval of Item and Associative Information. , 1982 .

[8]  Bennet B. Murdock,et al.  A distributed memory model for serial-order information. , 1983 .

[9]  Jon M. Slack.  A Parsing Architecture Based On Distributed Memory Machines , 1984, COLING.

[10]  Ray Pike,et al.  Comparison of convolution and matrix distributed memory systems for associative recall and recognition , 1984 .

[11]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[12]  J N Lee,et al.  Optical implementations of associative networks with versatile adaptive learning capabilities. , 1987, Applied optics.

[13]  Demetri Psaltis,et al.  Optical Associative Memory Using Fourier Transform Holograms , 1987 .

[14]  David S. Touretzky,et al.  A distributed connectionist representation for concept structures , 1987 .

[15]  Ronald Rosenfeld,et al.  Coarse-Coded Symbol Memories and Their Properties , 1988, Complex Syst..

[16]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[17]  Elke U. Weber,et al.  Expectation and variance of item resemblance distributions in a convolution-correction model of distributed memory , 1988 .

[18]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[19]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[20]  Douglas F. Elliott,et al.  Handbook of Digital Signal Processing: Engineering Applications , 1988 .

[21]  D Casasent,et al.  Key and recollection vector effects on heteroassociative memory performance. , 1989, Applied optics.

[22]  B. Murdock,et al.  Memory for Serial Order , 1989 .

[23]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[24]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[25]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[26]  Peter Dayan,et al.  Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down , 1990, Neural Computation.

[27]  David S. Touretzky,et al.  BoltzCONS: Dynamic Symbol Structures in a Connectionist Network , 1990, Artif. Intell..

[28]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[29]  Géraldine Legendre,et al.  Principles for an Integrated Connectionist/Symbolic Theory of Higher Cognition ; CU-CS-600-92 , 1992 .

[30]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[31]  Tony Plate,et al.  Holographic Recurrent Networks , 1992, NIPS.