Detecting systematic structure in distributed representations

A novel method of analysing the systematic structure formed inside a feedforward neural network is developed. This method is applied to the task of understanding how a network is performing unification on distributed representations. Systematic structure is detected by examining inter-representational distances, rather than by attempting to detect the vectorial similarities sought by previously described techniques. The method described can be applied to the detection of systematic structure in other networks using distributed representations.

[1]  Antony Browne,et al.  Neural Network Perspectives on Cognition and Adaptive Robotics , 1997 .

[2]  J. A. Robinson,et al.  A Machine-Oriented Logic Based on the Resolution Principle , 1965, JACM.

[3]  Brian Everitt,et al.  Cluster analysis , 1974 .

[4]  Renu Vig,et al.  Medical Diagnostic Expert System , 1993, HCI.

[5]  J. Fodor,et al.  Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work , 1990, Cognition.

[6]  Lars Niklasson,et al.  Connectionism and the Issues of Compositionality and Systematicity , 1992 .

[7]  David J. Chalmers,et al.  Syntactic Transformations on Distributed Representations , 1990 .

[8]  Lars Niklasson,et al.  Can Connectionist Models Exhibit Non-Classical Structure Sensitivity? , 2019, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[9]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[10]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[11]  Noel E. Sharkey.  The ghost in the hybrid-a study of uniquely connectionist representations , 1992 .

[12]  John A. Bullinaria,et al.  Analyzing the Internal Representations of Trained Neural Networks , 1997 .

[13]  Mikael Bodén,et al.  Representing Structure and Structured Representations in Connectionist Networks , 2019, Neural Network Perspectives on Cognition and Adaptive Robotics.

[14]  Lonnie Chrisman,et al.  Learning Recursive Distributed Representations for Holistic Computation , 1991 .

[15]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[16]  Antony Browne,et al.  Unification using a distributed representation , 1994, SGAR.

[17]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[18]  Noel E. Sharkey,et al.  An Internal Report for Connectionists , 1995 .

[19]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.