The psycholinguistics of signed and spoken languages: how biology affects processing

inguistic research over the last few decades has revealed substantial similarities between the structure of signed and spoken languages (for reviews see Emmorey, 2002; Sandler and Lillo-Martin, 2006). These similarities provide a strong basis for cross-modality comparisons, and also bring to light linguistic universals that hold for all human languages. In addition, however, biology-based distinctions between sign and speech are important, and can be exploited to discover how the input–output systems of language impact online language processing and affect the neurocognitive underpinnings of language comprehension and production. For example, do the distinct perceptual and productive systems of signed and spoken languages exert differing constraints on the nature of linguistic processing? Recent investigations have suggested that the modality in which a language is expressed can impact the psychological mechanisms required to decode and produce the linguistic signal. This chapter explores what aspects of language processing appear to be universal to all human languages and what aspects are affected by the particular characteristics of audition vs. vision, or by the differing constraints on manual versus oral articulation. Sign language processing is appropriately compared to speech processing, rather than to reading, because unlike written text, which can be characterized as “visual language,” sign language consists of dynamic and constantly changing forms rather than static symbols. Further, neither sign language nor spoken language comes pre-segmented into words and sentences for the perceiver. The production of writing, although performed by the hand, differs substantially from sign language production because writing derives its structure from a separate system (the orthography of a spoken language). In contrast to written language, sign and speech are both primary language systems, acquired during infancy and early childhood without formal instruction.

[1]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[2]  A. Liberman,et al.  The Identification and Discrimination of Synthetic Vowels , 1962 .

[3]  Albert T. Corbett,et al.  Pronoun disambiguation: Accessing potential antecedents , 1983, Memory & cognition.

[4]  Diane J. Schiano,et al.  Effects of Task and Object Configuration on Perspective Choice in Spatial Descriptions , 2002 .

[5]  Stephen R. Anderson,et al.  A-Morphous morphology , 1992 .

[6]  Elisabeth Engberg-Pedersen,et al.  Space in Danish sign language : the semantics and morphosyntax of the use of space in a visual language , 1993 .

[7]  L. Talmy.  Toward a Cognitive Semantics , 2003 .

[8]  Karen van Hoek,et al.  Conceptual Spaces and Pronominal Reference in American Sign Language , 1992, Nordic Journal of Linguistics.

[9]  Ursula Bellugi,et al.  Preliminaries to a distinctive feature analysis of handshapes in American Sign Language , 1976, Cognitive Psychology.

[10]  S. Goldinger,et al.  Priming Lexical Neighbors of Spoken Words: Effects of Competition and Inhibition. , 1989, Journal of memory and language.

[11]  Kearsy Cormier,et al.  Modality and structure in signed and spoken languages: Frontmatter , 2002 .

[12]  P. MacNeilage,et al.  The frame/content theory of evolution of speech production , 1998, Behavioral and Brain Sciences.

[13]  F Grosjean,et al.  Spoken word recognition processes and the gating paradigm , 1980, Perception & psychophysics.

[14]  M. Garrett.  Processes in language production , 1988 .

[15]  Diane Brentari,et al.  A Prosodic Model of Sign Language Phonology , 1999 .

[16]  T. Bever,et al.  Empty categories access their antecedents during comprehension , 1988 .

[17]  W. Levelt.  Models of word production , 1999, Trends in Cognitive Sciences.

[18]  Karen Emmorey,et al.  “Tip of the Fingers” Experiences by Deaf Signers , 2005, Psychological science.

[19]  Karen Emmorey,et al.  Mental rotation within linguistic and non-linguistic domains in users of American sign language , 1998, Cognition.

[20]  W. Sandler,et al.  On the nature of phonological structure in sign language , 1993, Phonology.

[21]  U. Zeshan.  Hand, head and face - negative constructions in sign languages , 2004 .

[22]  Willem J. M. Levelt,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[23]  B. C. Griffith,et al.  The discrimination of speech sounds within and across phoneme boundaries. , 1957, Journal of experimental psychology.

[24]  G. Dell,et al.  Lexical access in aphasic and nonaphasic speakers. , 1997, Psychological review.

[25]  Leonard Talmy,et al.  Toward a cognitive semantics, Vol. 1: Concept structuring systems. , 2000 .

[26]  Karen Emmorey,et al.  Conceptual Locations and Pronominal Reference in American Sign Language , 2004, Journal of psycholinguistic research.

[27]  M. F. Schober.  Spatial perspective-taking in conversation , 1993, Cognition.

[28]  F. Keil,et al.  Categorical effects in the perception of faces , 1995, Cognition.

[29]  W. Levelt,et al.  Monitoring and self-repair in speech , 1983, Cognition.

[30]  U. Bellugi,et al.  A comparison of sign language and spoken language , 1972 .

[31]  Victoria A. Fromkin,et al.  The Non-Anomalous Nature of Anomalous Utterances , 1971 .

[32]  Wendy Sandler,et al.  The Spreading Hand Autosegment of American Sign Language , 2013 .

[33]  Margaret Wilson,et al.  A “word length effect”for sign language: Further evidence for the role of language in structuring working memory , 1998, Memory & cognition.

[34]  Margaret Wilson,et al.  A visuospatial “phonological loop” in working memory: Evidence from American Sign Language , 1997, Memory & cognition.

[35]  Ronnie B. Wilbur,et al.  American sign language: Linguistic and applied dimensions , 1987 .

[36]  Karen Emmorey,et al.  10 Categorical Versus Gradient Properties of Classifier Constructions in ASL , 2003 .

[37]  K. Emmorey,et al.  Lexical Recognition in Sign Language: Effects of Phonetic Structure and Morphology , 1990, Perceptual and motor skills.

[38]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[39]  F. Newmeyer.  Linguistics: The Cambridge Survey , 1989 .

[40]  W. Marslen-Wilson.  Functional parallelism in spoken word-recognition , 1987, Cognition.

[41]  Alan S. Brown,et al.  A review of the tip-of-the-tongue experience. , 1991, Psychological bulletin.

[42]  Jack Gandour,et al.  Counterfeit tones in the speech of Southern Thai bidialectals , 1977 .

[43]  Sherman Wilcox,et al.  Gesture and the Nature of Language , 1995 .

[44]  W. Cooper,et al.  Sentence Processing: Psycholinguistic Studies Presented to Merrill Garrett. , 1980 .

[45]  Bennett L. Schwartz,et al.  Sparkling at the end of the tongue: The etiology of tip-of-the-tongue phenomenology , 1999 .

[46]  Roberta Michnick Golinkoff,et al.  The perception of handshapes in American Sign Language , 2005, Memory & cognition.

[47]  Janet Dean Fodor,et al.  Empty categories in sentence processing , 1989 .

[48]  Karen Emmorey,et al.  Processing spatial anaphora: Referent reactivation with overt and null pronouns in american sign language , 1995 .

[49]  L. Goldstein,et al.  Lexical retrieval in American Sign Language production , 2006 .

[50]  Bencie Woll,et al.  Sign Language: The Study of Deaf People and their Language , 1985 .

[51]  Linda Uyechi,et al.  The geometry of visual phonology , 1996 .

[52]  R. Battison,et al.  Lexical Borrowing in American Sign Language , 1978 .

[53]  W. Stokoe,et al.  Sign language structure: an outline of the visual communication systems of the American deaf. 1960. , 1961, Journal of deaf studies and deaf education.

[54]  M. F. Garrett,et al.  The Analysis of Sentence Production1 , 1975 .

[55]  M. D’Esposito.  Working memory. , 2008, Handbook of clinical neurology.

[56]  Karen Emmorey.  Non-antecedent Suppression in American Sign Language , 1997 .

[57]  David P. Corina.  TO BRANCH OR NOT TO BRANCH: UNDERSPECIFICATION IN ASL HANDSHAPE CONTOURS , 1993 .

[58]  J. Hawkins,et al.  The suffixing preference: a processing explanation , 1985 .

[59]  K. Emmorey.  Language, Cognition, and the Brain: Insights From Sign Language Research , 2001 .

[60]  Wendy Sandler,et al.  Sign Language and Linguistic Universals: Entering the lexicon: lexicalization, backformation, and cross-modal borrowing , 2006 .

[61]  F. Grosjean.  Sign & Word Recognition: A First Comparison , 2013 .

[62]  A. Postma,et al.  Production and Detection of Speech Errors in Silent, Mouthed, Noise-Masked, and Normal Auditory Feedback Speech , 1996 .

[63]  Richard P. Meier,et al.  Modality and structure in signed and spoken languages: Why different, why the same? Explaining effects and non-effects of modality upon linguistic structure in sign and speech , 2002 .

[64]  E. Klima.  The signs of language , 1979 .

[65]  William C Stokoe.  Semantic Phonology , 2001 .

[66]  Karen Emmorey,et al.  Spatial perspective choice in ASL , 2002 .

[67]  M. Aronoff,et al.  The Paradox of Sign Language Morphology , 2005, Language.

[68]  M. Gernsbacher.  Mechanisms that improve referential access , 1989, Cognition.

[69]  References , 1971 .

[70]  Ronnie B. Wilbur,et al.  SYLLABLES AND SEGMENTS: HOLD THE MOVEMENT AND MOVE THE HOLDS! , 1993 .

[71]  D. Corina,et al.  Phonological similarity in American Sign Language , 2002 .

[72]  Philip J. Benson,et al.  Categorical Perception of Facial Expressions: Categories and their Internal Structure , 1997 .

[73]  Bencie Woll,et al.  The Linguistics of British Sign Language: An Introduction , 1999 .

[74]  Peter A. Starreveld,et al.  On the Interpretation of Onsets of Auditory Context Effects in Word Production , 2000 .

[75]  Ardi Roelofs,et al.  Phonological Segments and Features as Planning Units in Speech Production , 1999 .

[76]  Ronnie B. Wilbur,et al.  American sign language: Linguistic and applied dimensions , 1987 .

[77]  Antje S. Meyer,et al.  Exploring the time course of lexical access in language production : Picture word interference studies , 1990 .

[78]  A. Hohenberger,et al.  Modality-dependent aspects of sign language production. Evidence from slips of the hands and their repairs in German Sign Language (DGS) , 2002 .

[79]  V. Fromkin.  Errors in linguistic performance: Slips of the tongue , 1982 .

[80]  Karen Emmorey,et al.  The activation of spatial antecedents from overt pronouns in american sign language , 1991 .

[81]  A. Hohenberger,et al.  The impact of modality on language production: Evidence from slips of the tongue and hand , 2004 .

[82]  D. Brentari,et al.  Categorical perception in American Sign Language , 2003 .

[83]  F. Grosjean,et al.  Sign Recognition Processes in American Sign Language: the Effect of Context , 1982 .

[84]  Gerilee Gustason,et al.  Signing Exact English , 1975 .