Categorical Perception of Face Actions: Their Role in Sign Language and in Communicative Facial Displays

Can face actions that carry significance within language be perceived categorically? We used continua produced by computational morphing of face-action images to explore this question in a controlled fashion. In Experiment 1 we showed that question-type—a syntactic distinction in British Sign Language (BSL)—can be perceived categorically, but only when it is also identified as a question marker. A few hearing non-signers were sensitive to this distinction; among those who used sign, late sign learners were no less sensitive than early sign users. A very similar facial-display continuum between “surprise” and “puzzlement” was perceived categorically by deaf and hearing participants, irrespective of their sign experience (Experiment 2). The categorical processing of facial displays can be demonstrated for sign, but may be grounded in universally perceived distinctions between communicative face actions. Moreover, the categorical perception of facial actions is not confined to the six universal facial expressions.

[1]  Russell Ultan,et al.  Some General Characteristics of Interrogative Systems. Working Papers on Language Universals, No. 1. , 1969 .

[2]  R. Feldman,et al.  Knowledge of American sign language and the ability of hearing individuals to decode facial expressions of emotion , 1996 .

[3]  U. Bellugi,et al.  Language, modality and the brain , 1989, Trends in Neurosciences.

[4]  Quinn McNemar,et al.  Psychological statistics, 2nd ed. , 1955 .

[5]  P R Killeen,et al.  Japanese quail can learn phonetic categories. , 1987, Science.

[6]  R. Mayberry,et al.  First-language acquisition after childhood differs from second-language acquisition: the case of American Sign Language. , 1993, Journal of speech and hearing research.

[7]  A. J. Fridlund.  Evolution and facial action in reflex, social motive, and paralanguage , 1991, Biological Psychology.

[8]  B. Rosner,et al.  Categories and boundaries in speech and music* , 1974 .

[9]  D. Pisoni.  Identification and discrimination of the relative onset time of two component tones: implications for voicing perception in stops. , 1977, The Journal of the Acoustical Society of America.

[10]  P. Ekman.  Unmasking The Face , 1975 .

[11]  Howard Poizner,et al.  Crosslinguistic/crossmodal syntactic consequences of left-hemisphere damage: Evidence from an aphasic signer and his identical twin , 1997 .

[12]  D. Perrett,et al.  Categorical Perception of Morphed Facial Expressions , 1996 .

[13]  Karen Emmorey,et al.  Enhanced Facial Diserimination: Effects of Experience With American Sign Language , 1997 .

[14]  U. Bellugi,et al.  A comparison of sign language and spoken language , 1972 .

[15]  Rachel I. Mayberry,et al.  The long-lasting advantage of learning sign language in childhood: Another look at the critical period for language acquisition , 1991 .

[16]  Ursula Bellugi,et al.  Faces: The Relationship Between Language and Affect , 1990 .

[17]  R. Woodworth,et al.  Experimental psychology, Rev. ed. , 1954 .

[18]  S. Harnad.  Category Induction and Representation , 1987 .

[19]  D. Povinelli,et al.  Mindblindness. An Essay on Autism and Theory of Mind Simon Baron-Cohen 1995 , 1996, Trends in Neurosciences.

[20]  John J. Magee,et al.  Categorical perception of facial expressions , 1992, Cognition.

[21]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[22]  P. Ekman,et al.  Pan-Cultural Elements in Facial Displays of Emotion , 1969, Science.

[23]  Bencie Woll.  The Semantics of British Sign Language Signs , 1983 .

[24]  J. Russell.  A circumplex model of affect. , 1980 .

[25]  D. Perrett,et al.  Facial expression megamix: Tests of dimensional and category accounts of emotion recognition , 1997, Cognition.

[26]  Patricia Siple,et al.  Understanding language through sign language research , 1978 .

[27]  B. C. Griffith,et al.  The discrimination of speech sounds within and across phoneme boundaries. , 1957, Journal of experimental psychology.

[28]  Philip J. Benson,et al.  Morph transformation of the facial image , 1994, Image Vis. Comput..

[29]  J D Miller,et al.  Speech perception by the chinchilla: identification function for synthetic VOT stimuli. , 1978, The Journal of the Acoustical Society of America.

[30]  P. Ekman.  The argument and evidence about universals in facial expressions of emotion. , 1989 .

[31]  Virginia Volterra,et al.  From Gesture to Language in Hearing and Deaf Children , 1990 .

[32]  David I. Perrett,et al.  Synthesising continuous-tone caricatures , 1991, Image Vis. Comput..

[33]  Philip J. Benson,et al.  Categorical Perception of Facial Expressions: Categories and their Internal Structure , 1997 .

[34]  F. Vargha-Khadem.  Visual field asymmetries in congenitally deaf and hearing children , 1983 .

[35]  A. O'Toole,et al.  Structural aspects of face recognition and the other-race effect , 1994, Memory & cognition.

[36]  S. Harnad.  Categorical Perception: The Groundwork of Cognition , 1990 .

[37]  F. Keil,et al.  Categorical effects in the perception of faces , 1995, Cognition.

[38]  J. Werker,et al.  The organization and reorganization of human speech perception. , 1992, Annual review of neuroscience.

[39]  Margaret Deuchar.  British Sign Language , 1984 .

[40]  Diane C. Lillo-Martin,et al.  Enhancement of Spatial Cognition in Deaf Children , 1990 .