Co-speech gesture in bimodal bilinguals

The effects of knowledge of sign language on co-speech gesture were investigated by comparing the spontaneous gestures of bimodal bilinguals (native users of American Sign Language and English; n=13) and non-signing native English speakers (n=12). Each participant viewed and re-told the Canary Row cartoon to a non-signer whom they did not know. Nine of the thirteen bimodal bilinguals produced at least one ASL sign, which we hypothesise resulted from a failure to inhibit ASL. Compared with non-signers, bimodal bilinguals produced more iconic gestures, fewer beat gestures, and more gestures from a character viewpoint. The gestures of bimodal bilinguals also exhibited a greater variety of handshape types and more frequent use of unmarked handshapes. We hypothesise that these semantic and form differences arise from an interaction between the ASL language production system and the co-speech gesture system.

[1]  Willem J. M. Levelt,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[2]  Marianne Gullberg,et al.  BIDIRECTIONAL CROSSLINGUISTIC INFLUENCE IN L1-L2 ENCODING OF MANNER IN SPEECH AND GESTURE: A Study of Japanese Speakers of English , 2008, Studies in Second Language Acquisition.

[3]  Sotaro Kita,et al.  The content of the message influences the hand choice in co-speech gestures and in gesturing without speaking , 2003, Brain and Language.

[4]  R. Schreuder,et al.  Producing words in a foreign language: Can speakers prevent interference from their first language? , 1998, Bilingualism: Language and Cognition.

[5]  Karen Emmorey,et al.  The psycholinguistics of signed and spoken languages: how biology affects processing , 2007 .

[6]  R. Battison,et al.  Lexical Borrowing in American Sign Language , 1978 .

[7]  Karen Emmorey,et al.  10 Categorical Versus Gradient Properties of Classifier Constructions in ASL , 2003 .

[8]  Lynn A. Friedman,et al.  Space, Time, and Person Reference in American Sign Language. , 1975 .

[9]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[10]  Gary L. Allen,et al.  Gestures Accompanying Verbal Route Directions: Do They Point to a New Avenue for Examining Spatial Representations? , 2003, Spatial Cogn. Comput..

[11]  Sotaro Kita,et al.  What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking , 2003 .

[12]  Albert Costa,et al.  How do highly proficient bilinguals control their lexicalization process? Inhibitory and language-specific selection mechanisms are both functional. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[13]  Scott K. Liddell.  Grammar, Gesture, and Meaning in American Sign Language , 2003 .

[14]  M. Alibali,et al.  Effects of Visibility between Speaker and Listener on Gesture Production: Some Gestures Are Meant to Be Seen , 2001 .

[15]  R Wesp,et al.  Gestures maintain spatial imagery. , 2001, The American journal of psychology.

[16]  Melanie Metzger,et al.  Gesture in sign language discourse , 1998 .

[17]  Karen Emmorey,et al.  Do signers gesture , 1999 .

[18]  Karen Emmorey,et al.  Bimodal bilingualism. , 2008, Bilingualism.

[19]  Gaurav Mathur,et al.  The morphology-phonology interface in signed languages , 2000 .

[20]  Albert Costa.  Lexical Access in Bilingual Production. , 2005 .

[21]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[22]  R. Krauss,et al.  PSYCHOLOGICAL SCIENCE Research Article GESTURE, SPEECH, AND LEXICAL ACCESS: The Role of Lexical Movements in Speech Production , 2022 .

[23]  K. Tuite.  The production of gesture , 1993 .

[24]  A. Caramazza,et al.  Lexical access in bilingual speakers: What's the (hard) problem? , 2006, Bilingualism: Language and Cognition.

[25]  Ezequiel Morsella,et al.  The role of gestures in spatial working memory and speech. , 2004, The American journal of psychology.

[26]  Karen Emmorey,et al.  The neural correlates of spatial language in English and American Sign Language: a PET study with hearing bilinguals , 2005, NeuroImage.

[27]  Susan Bell Trickett,et al.  The Relationship Between Spatial Transformations and Iconic Gestures , 2006, Spatial Cogn. Comput..

[28]  Sotaro Kita,et al.  How does linguistic framing of events influence co-speech gestures? , 2007 .

[29]  E. Klima.  The signs of language , 1979 .

[30]  J. Kroll,et al.  Language Production in Bilinguals , 1997 .

[31]  Diane Lillo-Martin,et al.  Modality and structure in signed and spoken languages: Where are all the modality effects? , 2002 .

[32]  Karen Emmorey,et al.  Perspectives on Classifier Constructions in Sign Languages , 2003 .

[33]  Kathryn Bock,et al.  Toward a Cognitive Psychology of Syntax: Information Processing Contributions to Sentence Formulation , 1982 .

[34]  Elena Nicoladis,et al.  A cross-cultural study on the use of gestures: Evidence for cross-linguistic transfer? , 2006 .

[35]  D. McNeill.  Hand and Mind: What Gestures Reveal about Thought , 1992 .

[36]  L. Messing,et al.  Two modes—two languages? , 1999 .

[37]  A. Colomé,et al.  Lexical Activation in Bilinguals' Speech Production: Language-Specific or Language-Independent? , 2001 .

[38]  Wendy Sandler,et al.  Sign Language and Linguistic Universals: Entering the lexicon: lexicalization, backformation, and cross-modal borrowing , 2006 .

[39]  M. Gullberg,et al.  BIDIRECTIONAL CROSSLINGUISTIC INFLUENCE IN L1-L2 ENCODING OF MANNER IN SPEECH AND GESTURE: A Study of Japanese Speakers of English , 2008, Studies in Second Language Acquisition.

[40]  Christian Rathmann,et al.  Modality and structure in signed and spoken languages: Is verb agreement the same crossmodally? , 2002 .

[41]  E A Franz,et al.  Bimanual gestures: Expressions of spatial representations that accompany speech processes , 2005, Laterality.

[42]  Pieter Muysken,et al.  Bilingual Speech: A Typology of Code-Mixing , 2000 .

[43]  Diane Brentari,et al.  Symmetry and dominance: A cross-linguistic study of signs and classifier constructions , 2007 .

[44]  P. B. Braem.  Acquisition of the Handshape in American Sign Language: A Preliminary Analysis , 1990 .