Compositionality in rational analysis: grammar-based induction for concept learning

Rational analysis attempts to explain aspects of human cognition as an adaptive response to the environment (Marr, 1982; Anderson, 1990; Chater, Tenenbaum, & Yuille, 2006). The dominant approach to rational analysis today takes an ecologically reasonable specification of a problem facing an organism, given in statistical terms, then seeks an optimal solution, usually using Bayesian methods. This approach has proven very successful in cognitive science; it has predicted perceptual phenomena (Geisler & Kersten, 2002; Feldman, 2001), illuminated puzzling effects in reasoning (Chater & Oaksford, 1999; Griffiths & Tenenbaum, 2006), and, especially, explained how human learning can succeed despite sparse input and endemic uncertainty (Tenenbaum, 1999; Tenenbaum & Griffiths, 2001). However, there were earlier notions of the “rational” analysis of cognition that emphasized very different ideas. One of the central ideas behind logical and computational approaches, which previously dominated notions of rationality, is that meaning can be captured in the structure of representations, but that compositional semantics are needed for these representations to provide a coherent account of thought. In this chapter we attempt to reconcile the modern approach to rational analysis with some aspects of this older, logico-computational approach. We do this via a model—offered as an extended example—of human concept learning. In the current chapter we are primarily concerned with formal aspects of this approach; in other work (Goodman, Tenenbaum, Feldman, & Griffiths, in press) we more carefully study a variant of this model as a psychological model of human concept learning. Explaining human cognition was one of the original motivations for the development of formal logic. George Boole, the father of digital logic, developed his symbolic language in order to explicate the rational laws underlying thought: his principal work, An Investigation of the Laws of Thought (Boole, 1854), was written to “investigate the fundamental laws of those operations of the mind by which reasoning is performed,” and arrived at “some probable intimations concerning the nature and constitution of the human mind” (p. 1). Much of mathematical logic since Boole can be regarded as an attempt to capture the coherence of thought in a formal system. This is particularly apparent in the work, by Frege (1892), Tarski (1956) and others, on model-theoretic semantics for logic, which aimed to create formal systems both flexible and systematic enough to capture the complexities of mathematical thought. A central component in this program is compositionality. Consider Frege’s Principle1: each syntactic operation of a formal language should have a corresponding semantic operation. This principle requires syntactic compositionality, that meaningful terms in a formal system are built up by combination operations, as well as compatibility between the syntax and semantics of the system. When Turing, Church, and others suggested that formal systems could be manipulated by mechanical computers it was natural (at least in hindsight) to suggest that cognition operates in a similar way: meaning is manipulated in the mind by computation2. Viewing the mind as a formal computational system in this way suggests that compositionality should also be found in the mind; that is, that mental representations may be combined into new representations, and the meaning of mental representations may be decomposed in terms of the meaning of their components. Two important virtues for a theory of thought result (Fodor, 1975): productivity—the number of representations is unbounded because they may be boundlessly combined— and systematicity—the combination of two representations is meaningful to one who can understand each separately. Despite its importance to the computational theory of mind, compositionality has seldom been captured by modern rational analyses. Yet there are a number of reasons to desire a compositional rational analysis. For instance, productivity of mental representations would provide an explanation of the otherwise puzzling ability of human thought to adapt to novel situations populated by new concepts—even those far beyond the ecological pressures of our evolutionary milieu (such as radiator repairs and the use of fiberglass bottom powerboats). We will show in this chapter that Bayesian statistical methods can be fruitfully combined with compositional representational systems by developing such a model in the well-studied setting of concept learning. This addresses a long running tension in the literature on human concepts: similarity-based statistical learning models have provided a good understanding of how simple concepts can be learned (Medin & Schaffer, 1978; Anderson, 1991; Kruschke, 1992;