Mental Simulation of Spatial Perspective during Sentence Comprehension

Mental Simulation of Spatial Perspective during Sentence Comprehension Sarah Schwarzkopf 1,2 (s.schwarzkopf@gmail.com), Helmut Weldle 2 , Daniel Muller 2 , Lars Konieczny 2,3 Department of Psychiatry, University of Cologne, Kerpener Str. 62 50924 Cologne, Germany Center for Cognitive Science, University of Freiburg, Friedrichstr. 50 79098 Freiburg, Germany Freiburg Institute for Advanced Studies, University of Freiburg, Albertstr. 19 79104 Freiburg, Germany Abstract We present an experiment, in which the Action-Sentence Compatibility Effect (ACE, Glenberg & Kaschak, 2002) was reversed when the perspective was changed by using first person pronouns as the agent or patient of the sentence. The results suggest that participants prefer to take the perspective of the first person protagonists – independent of their grammatical and semantic roles and the direction implicated by the action verb. We also discuss how mental simulations may work in sentences with only third person protagonists. Keywords: Perspective taking; models; ACE; sentence processing embodiment; situation Introduction When processing language, comprehenders generate a mental model with an inherent perspective on the described situation (Zwaan, 2004). Our goal is to find out how linguistic units, such as the verb or grammatical markers, influence the mental simulation during language processing. According to the embodied cognition hypothesis, language comprehension amounts to the mental simulation of the action involved, i.e. the activation of perceptual schemata and motoric programs. Empirically, the embodiment hypothesis received support from a vast body of evidence from both behavioral and neurofunctional studies (e.g. Aziz- Zadeh, Wilson, Rizzolatti & Iacoboni, 2006; Pulvermuller, Harle & Hummel, 2001; Zwaan, Stanfield & Yaxley, 2002). Glenberg and Kaschak (2002) reported the Action- Sentence Compatibility Effect (ACE): participants who had to respond to action-sentences, such as You open the drawer , by performing an arm-movement either directed towards their own body or in the opposite direction showed increased response latencies whenever the required direction of the arm movement was incompatible with the direction of the movement inherent in the sentence (sentence direction). Hence planning and execution of an incompatible movement appear to be inhibited while compatible movements are facilitated (Glenberg & Robertson, 1999). Processing sentences describing directed actions thus trigger the execution of corresponding real actions. To integrate the diversity of findings, Zwaan (2004) proposed the Immersed Experiencer Framework, which conceives language comprehension as situation simulation from the perspective of an observer who her/himself has a specific position in the situation model. According to this view, language comprehenders mentally simulate the reference situation as if they perceived a real-life situation (Zwaan, 2004). While doing so, they can follow the perspectives of protagonists in the story. Therefore it is easier to recall objects and events, if they are relevant for the protagonist than if they are not (Morrow, Bower & Greenspan, 1989). Perspective taking plays a crucial role in interpreting words (Black, Turner & Bower, 1979; Morrow & Clark, 1988; Pustejowsky, 1995; Sanford & Garrod, 1998) and texts (MacWhinney, 2008). In the sentence The mouse approached the fence the distance between agent and fence and the distance from which the situation is perceived by the observer (or simulated by the recipient) are both smaller than in the sentence The tractor approached the fence (Morrow & Clark, 1988). This difference indicates that different words suggest different perspectives. MacWhinney (2005) claims that discourse comprehension in general amounts to tracking multiple perspectives, a cognitive skill that has evolved from adaptations that supported the tracking of visual perspectives. Languages have evolved to provide perspective tracking devices, such as subjecthood in sentences (MacWhinney, 1987). Applying this idea to the ACE, one might expect that there is a strong tendency to simulate the action from the sentence subject's perspective, which happens to coincide with the agent inherent in the verbs. Since the verb is the linguistic element in a sentence that encodes these actions and a single verb can also implicate directions and evoke action simulations (Black, Turner & Bower, 1979; Chen & Bargh, 1999; Tseng & Bergen, 2005), it seems obvious that the implicated spatial direction of its described action (verb direction) controls which sentence direction is simulated by the reader during language processing. However, language processing is more than just word processing. So it is important to understand the link between the processing of content words and grammatical constructions (Bergen & Wheeler, 2010). Thus, the question arises how verb direction can be set in relation to the sentence and its reader. When looking at the ACE we find shorter reaction times when the action performed by the participants is compatible with the action described in the sentence. Since usually the acting person in a sentence is the subject and the participant her/himself is also acting, it seems to be the easiest alternative to take the subject's perspective. However, it remains an open question whether the simulated perspective is dominated by the perspective of the action's subject, the topic of the sentence, whether it is verb-