论文信息 - Measuring Representation

Measuring Representation

We present a measure of representation in neural networks that we call ‘R’, which is based on information theory. We show how R relates to an analysis of distributed representation, viz. a principal components analysis of activation space. Finally, we argue that R is well suited to measure representation in neural networks.

A. Hintze | L. Marstaller | Christoph Adami

[1] Zenon W. Pylyshyn,et al. Computation and Cognition: Toward a Foundation for Cognitive Science , 1984 .

[2] R. A. Brooks,et al. Intelligence without Representation , 1991, Artif. Intell..

[3] Andy Clark,et al. Associative Engines: Connectionism, Concepts, and Representational Change , 1993 .

[4] A. Clark. Being There: Putting Brain, Body, and World Together Again , 1996 .

[5] Terrence J. Sejnowski,et al. The Computational Brain , 1996, Artif. Intell..

[6] C. Adami,et al. Physical complexity of symbolic sequences , 1996, adap-org/9605002.

[7] Paul Smolensky,et al. Connectionist AI, symbolic AI, and the brain , 1987, Artificial Intelligence Review.

[8] J. Elman. Distributed representations, simple recurrent networks, and grammatical structure , 1991, Machine Learning.

[9] Sang Joon Kim,et al. A Mathematical Theory of Communication , 2006 .

[10] Gerard O'Brien,et al. The role of representation in computation , 2009, Cognitive Processing.

[11] Chris Thornton,et al. Representation Recovers Information , 2009, Cogn. Sci..