Rule induction in a neural network through integrated symbolic and subsymbolic processing

Two primary approaches to cognitive modeling have emerged in the last two decades: symbolic (or traditional) and subsymbolic (or connectionist). The symbolic approach characterizes cognition in terms of explicit rules phrased in a language of symbolic elements, e.g., if animal has sharp teeth and is big then run away fast. In contrast, the subsymbolic approach characterizes cognition in terms of nonlinear dynamical systems and numerical computation on state vectors; these complex computations are not readily reduced to rules; there is rarely a notion of discrete symbolic elements or antecedents and consequents. In the past these paradigms have been viewed as mutually exclusive. The main claim in this thesis is that the two paradigms have complementary strengths and can have synergistic interactions. Because many cognitive domains are fundamentally rule governed, abandoning the formalism provided by symbolic rules is unnecessary and unwise. However, most cognitive domains also involve some type of categorization to construct symbols (e.g., to recognize that a particular object is a "table"), and much category formation appears to be intrinsically a subsymbolic process. Thus, rather than adopting one paradigm or the other; what is needed is an integration of the two. This thesis presents a neural network architecture, called RuleNet, that learns both a functional categorization of input elements and rules based upon those categories. The rules perform string-to-string mappings of the general form: if certain conditions on the elements of the input string or the categorization of these elements are true then apply a certain transformation of the string. Simulations show that in this domain, by explicitly learning rules, RuleNet can outperform other networks, as well as outperform a symbolic-statistical approach. It is also shown that RuleNet can be applied to real cognitive domains, such as the problem of case-role assignment, in which syntactic constituents of a sentence are mapped to their underlying semantic roles. The methodology used to develop RuleNet is quite general and should be useful in designing architectures for a wide range of rule formalisms.