Does Language Shape the Production and Perception of Gestures? A Study on late Chinese-English Bilinguals' Conceptions about Time

Does Language Shape the Production and Perception of Gestures? A Study on late Chinese-English Bilinguals’ Conceptions about Time Yan Gu (yan.gu@tilburguniversity.edu) Lisette Mol (l.mol@tilburguniversity.edu) Marieke Hoetjes (m.w.hoetjes@tilburguniversity.edu) Marc Swerts (m.g.j.swerts@tilburguniversity.edu) Tilburg center for Cognition and Communication (TiCC), Tilburg University, the Netherlands Abstract Does language influence the production and perception of gestures? The metaphorical use of language in representing time is deeply interlinked with actions in space, such as gestures. In Chinese, speakers can talk and gesture about time as if it were horizontal, sagittal, or vertical. In English, speakers rarely employ the vertical plane. Two experiments showed that the verbal use of vertical spatial metaphors had an online influence on the production and perception of gestures by late Chinese-English bilinguals. Participants produced more vertical gestures when talking about time references by use of vertical spatial metaphors, e.g. ‘shang- zhōu’ (literally: ‘above week’, meaning ‘last week’), and they preferred vertical gestures to horizontal gestures when perceiving time references with vertical spatial metaphors. Gestures are not only shaped by the language specific conceptualisation, but are also sensitive to the changes in linguistic choices, both in production and perception. Keywords: gesture; space; time; metaphor; Chinese Introduction When representing abstract conceptions, people often recruit metaphors from more concrete domains, for instance, ‘time is space’ (Casasanto & Boroditsky, 2008). In all cultures people use space to represent time (Boroditsky, 2000). For example, English speakers can use spatial metaphors to talk about time such as ‘The future lies not too far ahead’, and ‘The day has been long’. In English, as well as in many other languages, speakers often conceptualise the past at the back (or leftwards), and the future at the front (or rightwards) (Calbris, 2008; Clark, 1973). However, this is not the case for all languages: for example, speakers of Aymara have the concept that ‘The future is behind’ (Nunez & Sweetser, 2006). The spatiotemporal thought can also be expressed in speakers’ co-speech metaphoric gestures. Typically, English speakers produce horizontal and sagittal gestures to indicate timelines (Casasanto & Jasmin, 2012). Interestingly, Chinese speakers can additionally talk about time vertically by employing vertical spatial metaphors of ‘上’ (shang: above) and ‘下’ (xia: below) to indicate the time conceptions of early and late. For example, ‘上周’ (shang zhōu) can literally be translated as ‘above week’, which means ‘last week’, while ‘下下周’ (xia xia zhōu) as ‘below below week’, referring to ‘the week after next week’. Due to the differences in use of spatial metaphors in time conceptions, Boroditsky (2001) argues that Chinese speakers may have a different conceptualisation of time than English speakers. Her argument is based on Slobin’s (1987) ‘thinking-for- speaking’ hypothesis, which proposes that speakers’ habitual thinking in a language will influence their cognitive attention and speech production - that is, the choices of what to focus on and how to say it. Specifically in Chinese, Boroditsky (2001) believes that the use of vertical spatial metaphors to talk about time shapes Chinese speakers’ language-specific conceptual schema. Interestingly, Chinese speakers can also gesture about time vertically. For instance, they can point upward to indicate the concept of an earlier time such as ‘last week’ or point downward to indicate that of a later time such as ‘next week’ (Chui, 2011; Gu, Mol, Hoetjes, & Swerts, 2013). One possible explanation to account for such a phenomenon could be embodiment, which proposes that conceptual representations are largely grounded in a sensorimotor medium motor, and representations are activated and often instantiated in the forms of gestures. However, given the language speakers are Chinese, one cannot preclude that Chinese lexical representations are activated which in turn influence the gestural patterns. Then the question arises whether the vertical gesturing about time results from Chinese speakers having different time conceptualisations in general (Boroditsky, 2001), or whether there is an additional online effect of how time is linguistically encoded in Chinese. In other words, the vertical gesturing about time conceptions can be due to (a) Chinese speakers’ vertical thinking of time (a language- specific conceptual schema); (b) the linguistic choices of the vertical words which drive the gestures; or (c) both the conceptual schema and the current linguistic choices. If the first explanation holds, then Chinese speakers will also gesture vertically when speaking English. That means their vertical gesturing will be unaffected by the language itself. However, if the vertical gesturing is not merely caused by the general conceptualisation of time, then it is likely to be immediately affected by the language. According to Kita & Ozyurek’s (2003) Interface Hypothesis, gestures not only reflect imagistic (spatio- motoric) representations of events, but also aid thinking- for-speaking. In their production model, the generation of a gesture is modulated by two forces: the spatio-motoric thinking from the Working Memory, and the linguistic encoding possibilities from the Message Generator, both of which interact with each other. The ultimate content of a gesture is determined by the Action Generator, which takes into account the two forms of thinking, such that gestures are adjusted to fit the verbalisation. For instance, Kita & Ozyurek (2003) found that a scene of a ‘Rolling Event’ can be expressed as ‘rolling down’ in English, with manner and path conflated into a single clause. Therefore, this information tends to be conflated in