Facial Expressions and the Evolution of the Speech Rhythm

In primates, different vocalizations are produced, at least in part, by making different facial expressions. Not surprisingly, humans, apes, and monkeys all recognize the correspondence between vocalizations and the facial postures associated with them. However, one major dissimilarity between monkey vocalizations and human speech is that, in the latter, the acoustic output and associated movements of the mouth are both rhythmic (in the 3- to 8-Hz range) and tightly correlated, whereas monkey vocalizations have a similar acoustic rhythmicity but lack the concommitant rhythmic facial motion. This raises the question of how we evolved from a presumptive ancestral acoustic-only vocal rhythm to the one that is audiovisual with improved perceptual sensitivity. According to one hypothesis, this bisensory speech rhythm evolved through the rhythmic facial expressions of ancestral primates. If this hypothesis has any validity, we expect that the extant nonhuman primates produce at least some facial expressions with a speech-like rhythm in the 3- to 8-Hz frequency range. Lip smacking, an affiliative signal observed in many genera of primates, satisfies this criterion. We review a series of studies using developmental, x-ray cineradiographic, EMG, and perceptual approaches with macaque monkeys producing lip smacks to further investigate this hypothesis. We then explore its putative neural basis and remark on important differences between lip smacking and speech production. Overall, the data support the hypothesis that lip smacking may have been an ancestral expression that was linked to vocal output to produce the original rhythmic audiovisual speech-like utterances in the human lineage.

[1]  R. Saunders,et al.  Longitudinal magnetic resonance imaging study of rhesus monkey brain development , 2006, The European journal of neuroscience.

[2]  P. MacNeilage,et al.  The frame/content theory of evolution of speech production , 1998, Behavioral and Brain Sciences.

[3]  E. Jarvis,et al.  Learned Birdsong and the Neurobiology of Human Language , 2004, Annals of the New York Academy of Sciences.

[4]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[5]  D. Poeppel,et al.  The neurophysiology and evolution of the speech rhythm , 2014 .

[6]  E. Clark,et al.  The Child's Path to Spoken Language. , 1994 .

[7]  S. Quartz,et al.  Human Insula Activation Reflects Risk Prediction Errors As Well As Risk , 2008, The Journal of Neuroscience.

[8]  Steven Greenberg,et al.  On the Possible Role of Brain Rhythms in Speech Perception: Intelligibility of Time-Compressed Speech with Periodic and Aperiodic Insertions of Silence , 2009, Phonetica.

[9]  G. Rizzolatti,et al.  ß Federation of European Neuroscience Societies Mirror , 2003 .

[10]  D. Lieberman,et al.  Hyoid and tongue surface movements in speaking and eating. , 2002, Archives of oral biology.

[11]  C A Moore,et al.  Does speech emerge from earlier appearing oral motor behaviors? , 1996, Journal of speech and hearing research.

[12]  Melanie Vitkovitch,et al.  Visible Speech as a Function of Image Quality: Effects of Display Parameters on Lipreading Ability , 1996 .

[13]  S. Kojima,et al.  Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes) , 2004, Animal Cognition.

[14]  Vittorio Gallese,et al.  Emotional and Social Behaviors Elicited by Electrical Stimulation of the Insula in the Macaque Monkey , 2011, Current Biology.

[15]  D. Oller,et al.  Final Syllable Lengthening (FSL) in infant vocalizations , 2003, Journal of Child Language.

[16]  Lisa A. Parr,et al.  Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition , 2004, Animal Cognition.

[17]  B. Stein The new handbook of multisensory processes , 2012 .

[18]  Stephen V. Shepherd,et al.  Facial Muscle Coordination in Monkeys during Rhythmic Facial Expressions and Ingestive Movements , 2012, The Journal of Neuroscience.

[19]  K. Saberi,et al.  Cognitive restoration of reversed speech , 1999, Nature.

[20]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[21]  E. Visalberghi,et al.  Facial Displays in Young Tufted Capuchin Monkeys (Cebus apella): Appearance, Meaning, Context and Target , 2007, Folia Primatologica.

[22]  Esther Thelen,et al.  Rhythmical behavior in infancy: An ethological perspective. , 1981 .

[23]  D. Lewkowicz Developmental changes in infants' visual response to temporal frequency. , 1985 .

[24]  Susan Bowsfield The Symbolic Species: The Co-Evolution of Language and the Brain , 2004 .

[25]  L. Parr,et al.  Facial musculature in the rhesus macaque (Macaca mulatta): evolutionary and functional contexts with comparisons to chimpanzees and humans , 2009, Journal of anatomy.

[26]  Bruno B Averbeck,et al.  Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex , 2006, The Journal of Neuroscience.

[27]  Koichiro Matsuo,et al.  Kinematic linkage of the tongue, jaw, and hyoid during eating and speech. , 2010, Archives of oral biology.

[28]  D. Oller The emergence of the speech capacity , 2000 .

[29]  C. Keysers,et al.  Towards a unifying neural theory of social cognition. , 2006, Progress in brain research.

[30]  R. Plomp,et al.  Effect of reducing slow temporal modulations on speech reception. , 1994, The Journal of the Acoustical Society of America.

[31]  Frank H. Guenther,et al.  An fMRI investigation of syllable sequence production , 2006, NeuroImage.

[32]  A. Malécot,et al.  Syllabic Rate and Utterance Length in French , 1972, Phonetica.

[33]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[34]  Zachary M. Smith,et al.  Chimaeric sounds reveal dichotomies in auditory perception , 2002, Nature.

[35]  D. B. Bender,et al.  Visual properties of neurons in inferotemporal cortex of the Macaque. , 1972, Journal of neurophysiology.

[36]  T. Bergman Speech-like vocalized lip-smacking in geladas , 2013, Current Biology.

[37]  D. Ostry,et al.  Control of jaw orientation and position in mastication and speech. , 1994, Journal of neurophysiology.

[38]  W. Zonneveld Syllables and segments : Alan Bell and Joan B. Hooper (eds.), North-Holland Linguistic Series 40. Papers from the Symposium on Segment Organization and the Syllable, Boulder, Colorado, October 21-23, 1977. North_Holland Publ. Co., Amsterdam, 1978 , 1980 .

[39]  Roy D. Patterson,et al.  Vocal-Tract Resonances as Indexical Cues in Rhesus Monkeys , 2007, Current Biology.

[40]  M. Arbib From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics , 2005, Behavioral and Brain Sciences.

[41]  Hermann Ackermann,et al.  The contribution of the insula to motor aspects of speech production: A review and a hypothesis , 2004, Brain and Language.

[42]  Elizabeth M. Brannon,et al.  Monkeys Match the Number of Voices They Hear to the Number of Faces They See , 2005, Current Biology.

[43]  Asif A. Ghazanfar,et al.  The evolution of speech: vision, rhythm, cooperation , 2014, Trends in Cognitive Sciences.

[44]  Stefan J. Kiebel,et al.  Simulation of talking faces in the human brain improves auditory speech recognition , 2008, Proceedings of the National Academy of Sciences.

[45]  Arlette Kolta,et al.  Brainstem circuits that control mastication: do they have anything to say during speech? , 2006, Journal of communication disorders.

[46]  D. B. Bender,et al.  Visual Receptive Fields of Neurons in Inferotemporal Cortex of the Monkey , 1969, Science.

[47]  B. Karmel,et al.  Correlation of infants' brain and behavior response to temporal changes in visual stimulation. , 1977, Psychophysiology.

[48]  Chris I. Baker,et al.  Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions , 2005, Journal of Cognitive Neuroscience.

[49]  Karl Zilles,et al.  Cortical Orofacial Motor Representation in Old World Monkeys, Great Apes, and Humans , 2004, Brain, Behavior and Evolution.

[50]  Rick Dale An Integrative Research Strategy for Exploring Synergies in Natural Language Performance , 2015 .

[51]  Gunnar Fant,et al.  Acoustic Theory Of Speech Production , 1960 .

[52]  E. Huber Evolution of Facial Musculature and Cutaneous Field of Trigeminus. Part II , 1930, The Quarterly Review of Biology.

[53]  Luc H. Arnal,et al.  Dual Neural Routing of Visual Facilitation in Speech Processing , 2009, The Journal of Neuroscience.

[54]  K. Zilles,et al.  Evolution of the brainstem orofacial motor system in primates: a comparative study of trigeminal, facial, and hypoglossal nuclei. , 2005, Journal of human evolution.

[55]  Asif A. Ghazanfar,et al.  Monkeys and Humans Share a Common Computation for Face/Voice Integration , 2011, PLoS Comput. Biol..

[56]  L. Parr,et al.  Influence of Social Context on the Use of Blended and Graded Facial Displays in Chimpanzees , 2005, International Journal of Primatology.

[57]  R. Andrew,et al.  The origin and evolution of the calls and facial expressions of the primates. , 1963 .

[58]  L. Vogt Individual Development And Evolution The Genesis Of Novel Behavior , 2016 .

[59]  Frédéric E. Theunissen,et al.  The Modulation Transfer Function for Speech Intelligibility , 2009, PLoS Comput. Biol..

[60]  R. Ringel,et al.  Task-specific organization of activity in human jaw muscles. , 1988, Journal of speech and hearing research.

[61]  C. Ross,et al.  Scaling of chew cycle duration in primates. , 2009, American journal of physical anthropology.

[62]  S. Fujii,et al.  The Role of Rhythm in Speech and Language Rehabilitation: The SEP Hypothesis , 2014, Front. Hum. Neurosci..

[63]  Roger W. Steeve,et al.  Babbling and chewing: Jaw kinematics from 8 to 22 months , 2010, J. Phonetics.

[64]  G. Westergaard,et al.  Auditory--visual cross-modal perception of communicative stimuli in tufted capuchin monkeys (Cebus apella). , 2005, Journal of experimental psychology. Animal behavior processes.

[65]  Jordan R. Green,et al.  Babbling, chewing, and sucking: oromandibular coordination at 9 months. , 2008, Journal of speech, language, and hearing research : JSLHR.

[66]  N. Logothetis,et al.  Neuroperception: Facial expressions linked to monkey calls , 2003, Nature.

[67]  R. A. Hinde,et al.  COMMUNICATION BY POSTURES AND FACIAL EXPRESSIONS IN THE RHESUS MONKEY (MACACA MULATTA) , 2009 .

[68]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[69]  Asif A Ghazanfar,et al.  Monkey lipsmacking develops like the human speech rhythm. , 2012, Developmental science.

[70]  O. Pascalis,et al.  Spontaneous voice–face identity matching by rhesus monkeys for familiar conspecifics and humans , 2011, Proceedings of the National Academy of Sciences.

[71]  C A Moore,et al.  Development of chewing in children from 12 to 48 months: longitudinal study of EMG patterns. , 1997, Journal of neurophysiology.

[72]  A. Levitt,et al.  Evidence for Language-Specific Rhythmic Influences in the Reduplicative Babbling of French-and English-Learning Infants , 1991, Language and speech.

[73]  Morgan L. Gustison,et al.  Derived vocalizations of geladas (Theropithecus gelada) and the evolution of vocal complexity in primates , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[74]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[75]  Oded Ghitza,et al.  Linking Speech Perception and Neurophysiology: Speech Decoding Guided by Cascaded Oscillators Locked to the Input Rhythm , 2011, Front. Psychology.

[76]  P. MacNeilage The origin of speech , 2008 .

[77]  Jeffery A. Jones,et al.  Neural processes underlying perceptual enhancement by visual speech gestures , 2003, Neuroreport.

[78]  D. Lieberman,et al.  Ontogeny of postnatal hyoid and larynx descent in humans. , 2001, Archives of oral biology.

[79]  J. Morton,et al.  Developmental Neurocognition: Speech and Face Processing in the First Year of Life , 2008 .

[80]  S. Suomi,et al.  Reciprocal Face-to-Face Communication between Rhesus Macaque Mothers and Their Newborn Infants , 2009, Current Biology.

[81]  S. Karlsson,et al.  Characteristics of Masticatory Mandibular Movements and Velocity in Growing Individuals and Young Adults , 1991, Journal of dental research.

[82]  T H Crystal,et al.  Segmental durations in connected speech signals: preliminary results. , 1982, The Journal of the Acoustical Society of America.

[83]  D. Lewkowicz,et al.  Heterochrony and Cross-Species Intersensory Matching by Infant Vervet Monkeys , 2009, PloS one.

[84]  Pedro Tiago Martins,et al.  Attention mechanisms and the mosaic evolution of speech , 2014, Front. Psychol..

[85]  Asif A Ghazanfar,et al.  Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. , 2009, Journal of neurophysiology.

[86]  Asif A Ghazanfar,et al.  Interactions between the Superior Temporal Sulcus and Auditory Cortex Mediate Dynamic Face/Voice Integration in Rhesus Monkeys , 2008, The Journal of Neuroscience.

[87]  Luiz A. Baccalá,et al.  Information theoretic interpretation of frequency domain connectivity measures , 2010, Biological Cybernetics.

[88]  W. Fitch,et al.  Vocal production in nonhuman primates: Acoustics, physiology, and functional constraints on “honest” advertisement , 1995, American journal of primatology.

[89]  T. Deacon The Symbolic Species: The Co-evolution of Language and the Brain , 1998 .

[90]  Kevin Murphy,et al.  Speech production: Wernicke, Broca and beyond. , 2002, Brain : a journal of neurology.

[91]  L. Fogassi,et al.  Neonatal Imitation in Rhesus Macaques , 2006, PLoS biology.

[92]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[93]  C. Sherwood Comparative anatomy of the facial motor nucleus in mammals, with an analysis of neuron numbers in primates. , 2005, The anatomical record. Part A, Discoveries in molecular, cellular, and evolutionary biology.

[94]  Asif A Ghazanfar,et al.  Dynamic, rhythmic facial expressions and the superior temporal sulcus of macaque monkeys: implications for the evolution of audiovisual speech , 2010, The European journal of neuroscience.

[95]  J. Rauschecker,et al.  Neurobiological roots of language in primate audition: common computational properties , 2015, Trends in Cognitive Sciences.

[96]  Phonetic Systems and Phonological Development , 1993 .

[97]  William K. Redican,et al.  Facial Expressions in Nonhuman Primates , 1975 .

[98]  P. Marler,et al.  The role of articulation in the production of rhesus monkey, Macaca mulatta, vocalizations , 1993, Animal Behaviour.

[99]  P. MacNeilage,et al.  The articulatory basis of babbling. , 1995, Journal of speech and hearing research.

[100]  L. Rosenblum Primate Behavior: Developments in Field and Laboratory Research , 1970 .

[101]  Takaaki Kuratate,et al.  Linking facial animation, head motion and speech acoustics , 2002, J. Phonetics.

[102]  Asif A Ghazanfar,et al.  Monkey visual behavior falls into the uncanny valley , 2009, Proceedings of the National Academy of Sciences.

[103]  Mary E. Beckman,et al.  Framing a socio-indexical basis for the emergence and cultural transmission of phonological systems , 2015, J. Phonetics.

[104]  D. Oller,et al.  Phrasing in prelinguistic vocalizations. , 1995, Developmental psychobiology.

[105]  Ruth E. Cumming,et al.  Awareness of Rhythm Patterns in Speech and Music in Children with Specific Language Impairments , 2015, Front. Hum. Neurosci..

[106]  Steven Greenberg,et al.  Temporal properties of spontaneous speech - a syllable-centric perspective , 2003, J. Phonetics.

[107]  Paul S. Katz,et al.  Homology and homoplasy of swimming behaviors and neural circuits in the Nudipleura (Mollusca, Gastropoda, Opisthobranchia) , 2012, Proceedings of the National Academy of Sciences.

[108]  Howard N Zelaznik,et al.  Development of functional synergies for speech motor coordination in childhood and adolescence. , 2004, Developmental psychobiology.

[109]  Yale E Cohen,et al.  Acoustic features of rhesus vocalizations and their representation in the ventrolateral prefrontal cortex. , 2007, Journal of neurophysiology.

[110]  A. Ghazanfar,et al.  The neurobiology of primate vocal communication , 2014, Current Opinion in Neurobiology.

[111]  C. Boeckx,et al.  Commentary on: Labels, cognomes, and cyclic computation: an ethological perspective , 2015, Front. Psychol..

[112]  N. Dronkers A new brain region for coordinating speech articulation , 1996, Nature.

[113]  A. Ghazanfar,et al.  Evolution of human vocal production , 2008, Current Biology.

[114]  M. Hauser,et al.  The Role of Lip Configuration in Monkey Vocalizations: Experiments Using Xylocaine as a Nerve Block , 1994, Brain and Language.

[115]  Christoph Kayser,et al.  Monkeys are perceptually tuned to facial expressions that exhibit a theta-like speech rhythm , 2013, Proceedings of the National Academy of Sciences.

[116]  K. Hiiemae,et al.  Tongue movements in feeding and speech. , 2003, Critical reviews in oral biology and medicine : an official publication of the American Association of Oral Biologists.

[117]  Peter F. MacNeilage,et al.  Characteristics of the rhythmic organization of vocal babbling: implications for an amodal linguistic rhythm. , 2008, Infant behavior & development.

[118]  Asif A. Ghazanfar,et al.  The Natural Statistics of Audiovisual Speech , 2009, PLoS Comput. Biol..

[119]  A. Ghazanfar,et al.  Cineradiography of Monkey Lip-Smacking Reveals Putative Precursors of Speech Dynamics , 2012, Current Biology.

[120]  Kathleen R. Gibson,et al.  Myelination and behavioral development: A comparative perspective on questions of neoteny, altriciality and intelligence. , 1991 .

[121]  Hani Yehia,et al.  Quantitative association of vocal-tract and facial behavior , 1998, Speech Commun..