Towards Bridging the Gap Between Pattern Recognition and Symbolic Representation Within Neural Networks

Underlying symbolic representations are opaque within neural networks that perform pattern recognition. Neural network weights are sub-symbolic, they commonly do not have a direct symbolic correlates. This work shows that by implementing network dynamics differently, during the testing phase instead of the training phase, pattern recognition can be performed using symbolically relevant weights. This advancement is an important step towards the merging of neural-symbolic representation, memory, and reasoning with pattern recognition.

[1]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[2]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[3]  Jürgen Schmidhuber,et al.  Learning Complex, Extended Sequences Using the Principle of History Compression , 1992, Neural Computation.

[4]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[5]  Fred Keijzer,et al.  Duality of the mind: a bottom up approach toward cognition , 2003 .

[6]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[7]  Stan Franklin,et al.  THE LIDA ARCHITECTURE: ADDING NEW MODES OF LEARNING TO AN INTELLIGENT, AUTONOMOUS, SOFTWARE AGENT , 2006 .

[8]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[9]  E. Oja,et al.  Independent Component Analysis , 2001 .

[10]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[11]  Eyal Amir,et al.  Counting objects with biologically inspired regulatory-feedback networks , 2009, 2009 International Joint Conference on Neural Networks.

[12]  Eyal Amir,et al.  Hybrid Classification and Symbolic-Like Manipulation Using Self-Regulatory Feedback Networks , 2008, NeSy.

[13]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[14]  Lokendra Shastri,et al.  Types and Quantifiers in SHRUTI: A Connectionist Model of Rapid Reasoning and Relational Processing , 1998, Hybrid Neural Systems.

[15]  Graham W. Taylor,et al.  Deconvolutional Networks for Feature Learning , 2010 .

[16]  Yun Peng,et al.  Local conditions for phase transitions in neural networks with variable connection strengths , 1993, Neural Networks.

[17]  Rafal Bogacz,et al.  BRAINN: A Connectionist Approach to Symbolic Reasoning , 1998, NC.

[18]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[19]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[20]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[21]  Yann LeCun,et al.  Large Scale Online Learning , 2003, NIPS.

[22]  John E. Laird,et al.  Extending the Soar Cognitive Architecture , 2008, AGI.

[23]  Luís M. A. Bettencourt,et al.  Evaluating the Contribution of Top-Down Feedback and Post-Learning Reconstruction , 2011, BICA.

[24]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .