Encoding techniques for complex information structures in connectionist systems

Two general information-encoding techniques called ‘relative-position encoding’ and ‘pattern-similarity association’ are presented. They are claimed to be a convenient basis for the connectionist implementation of complex, short-term information processing of the sort needed in commonsense reasoning, semantic/pragmatic interpretation of natural language utterances, and other types of high-level cognitive processing. The relationships of the techniques to other connectionist information-structuring methods, and also to methods used in computers, are discussed in detail. The rich interrelationships of these other connectionist and computer methods are also clarified. We detail the particular, simple forms that the relative-position encoding and pattern-similarity association techniques take in our own connectionist system, called Conposit, in order to clarify some issues and to provide evidence that the techniques are indeed useful in practice.

[1]  P. Smolensky.  On variable binding and the representation of symbolic structures in connectionist systems , 1987 .

[2]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[3]  Geoffrey E. Hinton.  A Parallel Computation that Assigns Canonical Object-Based Frames of Reference , 1981, IJCAI.

[4]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[5]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. , 1982, Psychological review.

[6]  Wendy G. Lehnert,et al.  Symbolic/Subsymbolic Sentence Analysi: Exploiting the Best of Two Worlds , 1988 .

[7]  L. Shastri,et al.  A Connectionist System for Rule Based Reasoning With Multi-Place Predicates and Variables , 1989 .

[8]  P. Johnson-Laird.  Mental models , 1989 .

[9]  Gray Clossman,et al.  A model of categorization and learning in a connectionist broadcast system , 1988 .

[10]  Michael G. Dyer,et al.  High-level Inferencing in a Connectionist Network , 1989 .

[11]  David S. Touretzky,et al.  A distributed connectionist representation for concept structures , 1987 .

[12]  Geoffrey E. Hinton.  Using fast weights to deblur old memories , 1987 .

[13]  Kai Hwang,et al.  Computer architecture and parallel processing , 1984, McGraw-Hill Series in computer organization and architecture.

[14]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[15]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[16]  Ms. Kathleen Jensen,et al.  PASCAL User Manual and Report , 1975, Lecture Notes in Computer Science.

[17]  Thomas A. Standish.  Data Structure Techniques , 1980 .

[18]  Geoffrey E. Hinton,et al.  Proceedings of the 1988 Connectionist Models Summer School , 1989 .

[19]  Niklaus Wirth,et al.  Programming in MODULA-2 (3rd corrected ed.) , 1985 .

[20]  Michael G. Dyer,et al.  Dynamic, Non-Local Role Bindings and Inferencing in a Localist Network for Natural Language Understanding , 1988, NIPS.

[21]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[22]  Michael G. Dyer,et al.  Storing and Generalizing Multiple Instances While Maintaining Knowledge-Level Parallelism , 1989, IJCAI.

[23]  Wayne A. Wickelgran.  Context-sensitive coding, associative memory, and serial order in (speech) behavior. , 1969 .

[24]  P. Dicke,et al.  Feature linking via stimulus-evoked oscillations: experimental results from cat visual cortex and functional implications from a network model , 1989, International 1989 Joint Conference on Neural Networks.

[25]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[26]  J. Barnden,et al.  Winner-take-all networks: Time-based versus activation-based mechanisms for various selection goals , 1990, IEEE International Symposium on Circuits and Systems.

[27]  Ronald L. Greene,et al.  Connectionist Hashed Associative Memory , 1991, Artif. Intell..

[28]  John A. Barnden,et al.  Overcoming Rule-Based Rigidity and Connectionist Limitations through Massively-Parallel Case-Based Reasoning , 1992, Int. J. Man Mach. Stud..

[29]  David S. Touretzky,et al.  BoltzCONS: Dynamic Symbol Structures in a Connectionist Network , 1990, Artif. Intell..

[30]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[31]  Geoffrey E. Hinton.  Preface to the Special Issue on Connectionist Symbol Processing , 1990 .

[32]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[33]  James L. McClelland.  The programmable blackboard model of reading , 1986 .

[34]  James A. Hendler,et al.  Marker‐passing over Microfeatures: Towards a Hybrid Symbolic/Connectionist Model , 1989 .

[35]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.