Holistic Symbol Processing and the Sequential RAAM: An Evalutation

In recent years connectionist researchers have demonstrated many examples of holistic symbol processing, where symbolic structures are operated upon as a whole by a neural network, by using a connectionist compositional representation of the structures. In this paper the ability of the Sequential RAAM (SRAAM) to generate representations that support holistic symbol processing is evaluated by attempting to perform Chalmers’ syntactic transformations using it. It is found that the SRAAM requires a much larger hidden layer for this task than the RAAM and that it tends to cluster its hidden layer states close together, leading to a representation that is fragile to noise. The lessons for connectionism and holistic symbol processing are discussed and possible methods for improving the SRAAM’s performance suggested.

[1]  Imre Laszlo Balogh.  An analysis of a connectionist internal representation: do RAAM networks produce truly distributed representations? , 1993 .

[2]  R. French.  Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference , 2019, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[3]  James Alistair Hammerton.  Exploiting holistic computation : an evaluation of the sequential RAAM , 1999 .

[4]  Noel E. Sharkey.  Connectionist Natural Language Processing: Readings from Connection Science , 1992 .

[5]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[6]  Barry L. Kalman,et al.  Tail-recursive Distributed Representations and Simple Recurrent Networks , 1995 .

[7]  Dominic Palmer-Brown,et al.  (S)RAAM: An Analytical Technique for Fast and Reliable Derivation of Connectionist Symbol Structure Representations , 1997, Connect. Sci..

[8]  Geoffrey E. Hinton,et al.  Distributed representations and nested compositional structure , 1994 .

[9]  Douglas S. Blank,et al.  Exploring the Symbolic/Subsymbolic Continuum: A case study of RAAM , 1992 .

[10]  David J. Chalmers,et al.  Syntactic Transformations on Distributed Representations , 1990 .

[11]  Ashwin Ram,et al.  Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society , 2019 .

[12]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[13]  Lonnie Chrisman,et al.  Learning Recursive Distributed Representations for Holistic Computation , 1991 .

[14]  Barry L. Kalman,et al.  High performance training of feedforward and simple recurrent networks , 1997, Neurocomputing.

[15]  Georg Dorffner.  Neural Networks and a New Artificial Intelligence , 1999 .

[16]  Lai-Wan Chan,et al.  Confluent Preorder Parser as Finite State Automata , 1996, ICANN.

[17]  James Alistair Hammerton,et al.  Holistic Computation: Reconstructing a Muddled Concept , 1998, Connect. Sci..