Simulated Action in an Embodied Construction Grammar

Simulated Action in an Embodied Construction Grammar Benjamin Bergen (bergen@hawaii.edu) Dept of Linguistics, 1890 East-West Hall, 569 Moore Hall Honolulu, HI 96822 Nancy Chang (nchang@icsi.berkeley.edu) Shweta Narayan (shweta@icsi.berkeley.edu) International Computer Science Institute, 1947 Center St., Suite 600 Berkeley, CA 94704-1198, USA experience. There is strong evidence, seen below, that such embodied knowledge is automatically and unconsciously brought to bear during language understanding. Moreover, language users naturally make a broad range of associative and causal inferences based on language, a process not easily represented in an abstract symbol system. Conversely, a theory of linguistic meaning cannot be based on perceptuo-motor information alone. Linguistic units can be combined in ways that are not strictly predictable from their semantic properties. Our ability to judge the grammaticality of sentences like Chomsky’s (1957) classic Colorless green ideas sleep furiously example provides strong evidence of linguistic structure distinct from motor, perceptual, or other world knowledge. Additionally, our ability to understand sentences like My pet chicken kissed me on the cheek (even though chickens don’t have lips, presumably a prerequisite for kissing) shows that grounded motor knowledge does not suffice to account for our ability to extract meaning from language. One concrete solution to the drawbacks of purely abstract and purely perceptuo-motor approaches is to characterize mental representations as schematizations over modal knowledge (Fillmore 1982; Langacker 1987; Lakoff 1987; Barsalou 1999; Talmy 2000). This compromise view retains the best of both worlds: while language use involves the activation of perceptual and motor mechanisms, linguistic units themselves need only refer to schematic representations of these mechanisms. Proposals along these lines have inspired work investigating how the perceptual and motor structures underlying word meaning might be represented and schematized in computational models of human language processing (Regier 1996; Bailey 1997; Narayanan 1997). But the nature of the lexical and grammatical units that link these structures with linguistic forms has not yet been articulated precisely enough to support formal or computational implementation. This paper synthesizes diverse evidence for an integrated view of language use and presents Embodied Construction Grammar (ECG), a formally specified instantiation of the approach. We begin by surveying evidence of the importance of perceptual and motor simulation in higher- level cognition, especially in language use. We then briefly outline the ECG formalism and show how it supports a model of human language use in which linguistic meanings serve to parameterize motor and perceptual structures. The remainder of the paper presents two kinds of support for the model. First, we describe a pair of verb matching studies Abstract Various lines of research on language have converged on the premise that linguistic knowledge has as its basic unit pairings of form and meaning. The precise nature of the meanings involved, however, remains subject to the longstanding debate between proponents of arbitrary, abstract representations and those who argue for more detailed perceptuo-motor representations. We propose a model, Embodied Construction Grammar (ECG), which integrates these two positions by casting meanings as schematic representations embodied in human perceptual and motor systems. On this view, understanding everyday language entails running mental simulations of its perceptual and motor content. Linguistic meanings are parameterizations of aspects of such simulations; they thus serve as an interface between the relatively discrete properties of language and the detailed and encyclopedic knowledge needed for simulation. This paper assembles evidence from neural imaging and psycholinguistic experiments supporting this general approach to language understanding. It also introduces ECG as a model that fulfills the requisite constraints, and presents two kinds of support for the model. First, we describe two verbal matching studies that test predictions the model makes about the degree of motor detail available in lexical representations. Second, we demonstrate the viability and utility of ECG as a grammar formalism through its capacity to support computational models of language understanding and acquisition. Introduction Many theories of language take the basic unit of linguistic knowledge to be pairings of form and meaning, known as symbols or constructions (de Saussure 1916; Pollard & Sag 1994; Goldberg 1995; Langacker 1987). This view stems from the simple observation that language serves to convey meaning, using form. A speaker must thus know what linguistic forms are appropriate to encode the meanings s/he wishes to convey, and vice versa for an understander. The nature of the meaning representations of linguistic units, however, remains very much at issue. Suggestions in the literature range from relatively abstract representations, including both feature structures (Pollard & Sag 1994) and logical representations (May 1985), to more concrete perceptual- or motor-based representations (Langacker 1987; Barsalou 1999; Glenberg & Robertson 2000). Each of these approaches faces difficulties. Abstract symbol systems, whether feature-based or logical, invite the question of how (or even whether) they are ultimately linked to human perceptual, motor, affective, and other sorts of