Spatiotemporal coupling between speech and manual motor actions

Much evidence has been found for pervasive links between the manual and speech motor systems, including evidence from infant development, deictic pointing, and repetitive tapping and speaking tasks. We expand on the last of these paradigms to look at intra- and cross-modal effects of emphatic stress, as well as the effects of coordination in the absence of explicit rhythm. In this study, subjects repeatedly tapped their finger and synchronously repeated a single spoken syllable. On each trial, subjects placed an emphatic stress on one finger tap or one spoken syllable. Results show that both movement duration and magnitude are affected by emphatic stress regardless of whether that stress is in the same domain (e.g., effects on the oral articulators when a spoken repetition is stressed) or across domains (e.g., effects on the oral articulators when a tap is stressed). Though the size of the effects differs between intra-and cross-domain emphases, the implementation of stress affects both motor domains, indicating a tight connection. This close coupling is seen even in the absence of stress, though it is highlighted under stress. The results of this study support the idea that implementation of prosody is not domain-specific but relies on general aspects of the motor system.

[1]  S. Tasko,et al.  Association of orofacial muscle activity and movement during changes in speech rate and intensity. , 2003, Journal of speech, language, and hearing research : JSLHR.

[2]  H. Nagasaki,et al.  Correlations of stress and timing in periodic tapping , 1987 .

[3]  J. Morton,et al.  Perceptual centers (P-centers). , 1976 .

[4]  J. Abbs,et al.  Sensorimotor characteristics of speech motor sequences , 2004, Experimental Brain Research.

[5]  P. Ladefoged Three areas of experimental phonetics , 1967 .

[6]  Dwight Warren Miles Preferred Rates in Rhythmic Response , 1937 .

[7]  Susan Shaiman,et al.  Effects of prosody and position on the timing of deictic gestures. , 2013, Journal of speech, language, and hearing research : JSLHR.

[8]  Maurizio Gentilucci,et al.  Execution and observation of bringing a fruit to the mouth affect syllable pronunciation , 2004, The European journal of neuroscience.

[9]  Louis Goldstein,et al.  A task-dynamic toolkit for modeling the effects of prosodic structure on articulation , 2008, Speech Prosody 2008.

[10]  Shrikanth S. Narayanan,et al.  Timing effects of syllable structure and stress on nasals: A real-time MRI examination , 2009, J. Phonetics.

[11]  George D. Allen,et al.  Speech Rhythm: Its Relation to Performance Universals and Articulatory Timing. , 1975 .

[12]  S. Tasko,et al.  Association of orofacial with laryngeal and respiratory motor output during speech , 2002, Experimental Brain Research.

[13]  J. Summers,et al.  Timing in finger tapping and speech: A comparison between stutterers and fluent speakers☆ , 1992 .

[14]  Cécile Fougeron,et al.  Articulatory properties of initial segments in several prosodic constituents in French , 2001, J. Phonetics.

[15]  Taehong Cho,et al.  Domain-initial articulatory strengthening in four languages , 2003 .

[16]  Susan Shaiman,et al.  Effects of perturbation and prosody on the coordination of speech and gesture , 2014, Speech Commun..

[17]  M A Rogers,et al.  Inter-limb coupling in coordinated bimanual movement: attention and asymmetries. , 1998, Laterality.

[18]  V. Gracco,et al.  Speech motor coordination and control: evidence from lip, jaw, and laryngeal movements , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  J. Iverson,et al.  The relationship between reduplicated babble onset and laterality biases in infant rhythmic arm movements , 2007, Brain and Language.

[20]  M. Dohen,et al.  Pointing is 'special' , 2009 .

[21]  K. D. Jong The supraglottal articulation of prominence in English: Linguistic stress as localized hyperarticulation , 1995 .

[22]  J. Kelso,et al.  Functionally specific articulatory cooperation following jaw perturbations during speech: evidence for coordinative structures. , 1984, Journal of experimental psychology. Human perception and performance.

[23]  Beckman,et al.  Phonological Structure and Phonetic Form: Articulatory evidence for differentiating stress categories , 1994 .

[24]  William J. Hardcastle,et al.  Some phonetic and syntactic constraints on lingual coarticulation during /kl/ sequences , 1985, Speech Commun..

[25]  Anders Löfqvist,et al.  Dynamics of intergestural timing: a perturbation study of lip-larynx coordination , 1998, Experimental Brain Research.

[26]  J A Kelso,et al.  Lip-larynx coordination in speech: effects of mechanical perturbations to the lower lip. , 1994, The Journal of the Acoustical Society of America.

[27]  Dani Byrd,et al.  Articulatory evidence for functional coupling of speech and non-speech motor tasks , 2010 .

[28]  F. Huddle Coordination , 1966, Open Knowledge Institutions.

[29]  Will Spijkers,et al.  Structural Constraints on the Performance of Symmetrical Bimanual Movements with Different Amplitudes , 1995 .

[30]  D K Oller,et al.  An event-based analysis of the coordination of early infant vocalizations and facial actions. , 1999, Developmental psychology.

[31]  A. Kristofferson,et al.  Response delays and the timing of discrete motor responses , 1973 .

[32]  Keiko Ejiri,et al.  Relationship between Rhythmic Behavior and Canonical Babbling in Infant Vocal Development , 1998, Phonetica.

[33]  Shrikanth S. Narayanan,et al.  Phrasal signatures in articulation , 2000 .

[34]  J. Iverson,et al.  The interplay between language, gesture, and affect during communicative transition: a dynamic systems approach. , 2011, Developmental psychology.

[35]  M. Wiesendanger,et al.  Coordination of bowing and fingering in violin playing. , 2005, Brain research. Cognitive brain research.

[36]  M. Beckman,et al.  The articulatory kinematics of final lengthening. , 1991, The Journal of the Acoustical Society of America.

[37]  Peter J. Beek,et al.  Relative phase dynamics in perturbed interlimb coordination: the effects of frequency and amplitude , 2000, Biological Cybernetics.

[38]  E. Thelen,et al.  Hand, mouth and brain. The dynamic emergence of speech and gesture , 1999 .

[39]  Amalia Arvaniti,et al.  The usefulness of metrics in the quantification of speech rhythm , 2012, J. Phonetics.

[40]  Jean-Luc Schwartz,et al.  The speech focus position effect on jaw-finger coordination in a pointing task. , 2008, Journal of speech, language, and hearing research : JSLHR.

[41]  G. Hammond,et al.  Mutual interactions between speech and finger movements. , 1987, Journal of motor behavior.

[42]  A. Williams,et al.  Local stability in coordinated rhythmic movements: fluctuations and relaxation times. , 2002, Human movement science.

[43]  Paul Treffner,et al.  Intentional and attentional dynamics of speech-hand coordination. , 2002, Human movement science.

[44]  G. Butterworth,et al.  Hand-mouth coordination in the new-born baby , 1988 .

[45]  D. Liberman,et al.  [Time perception]. , 1955, Revista de psicoanalisis.

[46]  Christine F Delgado,et al.  The temporal coordination of early infant communication. , 2003, Developmental psychology.

[47]  Taehong Cho,et al.  Domain-initial strengthening as enhancement of laryngeal features: Aerodynamic evidence from Korean , 2000 .

[48]  Peter J Beek,et al.  Handedness-related asymmetry in coupling strength in bimanual coordination: furthering theory and evidence. , 2007, Acta psychologica.

[49]  Louis Goldsteina,et al.  SYLLABLE STRUCTURE AS COUPLED OSCILLATOR MODES : EVIDENCE FROM GEORGIAN VS . TASHLHIYT BERBER , 2007 .

[50]  Nobuo Masataka,et al.  Co‐occurences of preverbal vocal behavior and motor action in early infancy , 2001 .

[51]  Willem J. M. Levelt,et al.  Pointing and voicing in deictic expressions , 1985 .

[52]  A Smith,et al.  Interactions between speech and finger movements: an exploration on the dynamic pattern perspective. , 1986, Journal of speech and hearing research.

[53]  M. Studdert-Kennedy Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[54]  C. Fowler Converging sources of evidence on spoken and perceived rhythms of speech: cyclic production of vowels in monosyllabic stress feet. , 1983, Journal of experimental psychology. General.

[55]  Núria Esteve-Gibert,et al.  Prosodic structure shapes the temporal realization of intonation and manual gesture movements. , 2013, Journal of speech, language, and hearing research : JSLHR.

[56]  Sungbok Lee,et al.  How far, how long: on the temporal scope of prosodic boundary effects. , 2006, The Journal of the Acoustical Society of America.

[57]  D. Abercrombie,et al.  Elements of General Phonetics , 1967 .

[58]  L. Max,et al.  Accuracy and variability of isochronous rhythmic timing across motor systems in stuttering versus nonstuttering individuals. , 2003, Journal of speech, language, and hearing research : JSLHR.

[59]  K J de Jong,et al.  The correlation of P-center adjustments with articulatory and acoustic events. , 1994, Perception & psychophysics.

[60]  E. Saltzman,et al.  Steady-state and perturbed rhythmical movements: a dynamical analysis. , 1991, Journal of experimental psychology. Human perception and performance.

[61]  S. Keele,et al.  Timing Functions of The Cerebellum , 1989, Journal of Cognitive Neuroscience.

[62]  H. Haken,et al.  A theoretical model of phase transitions in human hand movements , 2004, Biological Cybernetics.

[63]  K. D. de Jong The supraglottal articulation of prominence in English: linguistic stress as localized hyperarticulation. , 1995, The Journal of the Acoustical Society of America.

[64]  Nobuyuki Inui,et al.  Interactions of Speech and Manual Movement in a Syncopated Task , 2007, Perceptual and motor skills.

[65]  D P Carey,et al.  Eye–hand coordination: Eye to hand or hand to eye? , 2000, Current Biology.

[66]  Betty Tuller,et al.  A “Dynamic Pattern” Perspective on the Control and Coordination of Movement , 1983 .

[67]  J. Kelso,et al.  Nonequilibrium phase transitions in coordinated biological motion: Critical slowing down and switching time , 1987 .

[68]  P. Keating,et al.  Articulatory strengthening at edges of prosodic domains. , 1997, The Journal of the Acoustical Society of America.

[69]  Dani Byrd,et al.  Intragestural dynamics of multiple prosodic boundaries , 1998 .

[70]  Dani Byrd,et al.  The elastic phrase: modeling the dynamics of boundary-adjacent lengthening , 2003, J. Phonetics.

[71]  J. Iverson,et al.  Infant vocal-motor coordination: precursor to the gesture-speech system? , 2004, Child development.