Eye activity tracks task-relevant structures during speech and auditory sequence perception

The sensory and motor systems jointly contribute to complex behaviors, but whether motor systems are involved in high-order perceptual tasks such as speech and auditory comprehension remain debated. Here, we show that ocular muscle activity is synchronized to mentally constructed sentences during speech listening, in the absence of any sentence-related visual or prosodic cue. Ocular tracking of sentences is observed in the vertical electrooculogram (EOG), whether the eyes are open or closed, and in eye blinks measured by eyetracking. Critically, the phase of sentence-tracking ocular activity is strongly modulated by temporal attention, i.e., which word in a sentence is attended. Ocular activity also tracks high-level structures in non-linguistic auditory and visual sequences, and captures rapid fluctuations in temporal attention. Ocular tracking of non-visual rhythms possibly reflects global neural entrainment to task-relevant temporal structures across sensory and motor areas, which could serve to implement temporal attention and coordinate cortical networks.Our eyes constantly follow objects we see, but do they also move in synchrony with auditory inputs? Here, the authors show that eyelid movements track the temporal structure of speech and other sound sequences, which could reflect a role of motor systems in temporal attention and sequence processing.

[1]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[2]  Keiichi Kitajo,et al.  Synchronization of spontaneous eyeblinks while viewing video stories , 2009, Proceedings of the Royal Society B: Biological Sciences.

[3]  J C Rothwell,et al.  New observations on the normal auditory startle reflex in man. , 1991, Brain : a journal of neurology.

[4]  Michael J. Spivey,et al.  Eye movements during listening reveal spontaneous grammatical processing , 2014, Front. Psychol..

[5]  Isabelle Peretz,et al.  Tagging the Neuronal Entrainment to Beat and Meter , 2011, The Journal of Neuroscience.

[6]  N. Geschwind Language and the brain. , 1972, Scientific American.

[7]  M. Iacoboni,et al.  Listening to speech activates motor areas involved in speech production , 2004, Nature Neuroscience.

[8]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[9]  M. R. Jones,et al.  Dynamic attending and responses to time. , 1989, Psychological review.

[10]  Charles E. Schroeder,et al.  Motor contributions to the temporal precision of auditory attention , 2014, Nature Communications.

[11]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[12]  W. T. Fitch,et al.  The Evolution of Rhythm Processing , 2018, Trends in Cognitive Sciences.

[13]  L. Trainor,et al.  Feeling the Beat: Movement Influences Infant Rhythm Perception , 2005, Science.

[14]  Michael Schluroff Pupil responses to grammatical complexity of sentences , 1982, Brain and Language.

[15]  R. Rafal,et al.  Neural Control of Voluntary Eye Closure: A Case Study and an fMRI Investigation of Blinking and Winking , 2012, Behavioural neurology.

[16]  Asif A. Ghazanfar,et al.  The Natural Statistics of Audiovisual Speech , 2009, PLoS Comput. Biol..

[17]  T. Paus,et al.  Seeing and hearing speech excites the motor system involved in speech production , 2003, Neuropsychologia.

[18]  P. Fries Rhythms for Cognition: Communication through Coherence , 2015, Neuron.

[19]  W. Marslen-Wilson Accessing Spoken Words : The Importance of Word Onsets , 2004 .

[20]  Lisa D. Sanders,et al.  An ERP study of continuous speech processing. I. Segmentation, semantics, and syntax in native speakers. , 2003, Brain research. Cognitive brain research.

[21]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[22]  J. Obleser,et al.  Spatiotemporal dynamics of auditory attention synchronize with speech , 2016, Proceedings of the National Academy of Sciences.

[23]  R. Zatorre,et al.  Listening to musical rhythms recruits motor regions of the brain. , 2008, Cerebral cortex.

[24]  W. Marslen-Wilson,et al.  Accessing Spoken Words: The Importance of Word Onsets , 1989 .

[25]  Peter Lakatos,et al.  Dynamics of Active Sensing and perceptual selection , 2010, Current Opinion in Neurobiology.

[26]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[27]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[28]  François Christophe Egidio Pellegrino,et al.  Across-Language Perspective on Speech Information Rate , 2011 .

[29]  G. Rizzolatti,et al.  Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention , 1987, Neuropsychologia.

[30]  Rufin VanRullen,et al.  On the cyclic nature of perception in vision versus audition , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[31]  Marco Iacoboni,et al.  The Essential Role of Premotor Cortex in Speech Perception , 2007, Current Biology.

[32]  Luc H. Arnal,et al.  Cortical oscillations and sensory predictions , 2012, Trends in Cognitive Sciences.

[33]  S. Steinhauer,et al.  Blink before and after you think: blinks occur prior to and following cognitive load indexed by pupillary responses. , 2008, Psychophysiology.

[34]  A. Sanford,et al.  Depth of processing in language comprehension: not noticing the evidence , 2002, Trends in Cognitive Sciences.

[35]  Haitao Liu,et al.  Dependency distance: A new perspective on syntactic patterns in natural languages. , 2017, Physics of life reviews.

[36]  Stanislas Dehaene,et al.  Neurophysiological dynamics of phrase-structure building during sentence processing , 2017, Proceedings of the National Academy of Sciences.

[37]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[38]  J. Simon,et al.  Emergence of neural encoding of auditory objects while listening to competing speakers , 2012, Proceedings of the National Academy of Sciences.

[39]  M. Pickering,et al.  An integrated theory of language production and comprehension. , 2013, The Behavioral and brain sciences.

[40]  Iwona Stepniewska,et al.  Microstimulation reveals specialized subregions for different complex movements in posterior parietal cortex of prosimian galagos. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Riikka Möttönen,et al.  Auditory-Motor Processing of Speech Sounds , 2012, Cerebral cortex.

[42]  Dong Pyo Jang,et al.  Correlation between Inter-Blink Interval and Episodic Encoding during Movie Watching , 2015, PloS one.

[43]  Yinjuan Du,et al.  Noise differentially impacts phoneme representations in the auditory and speech motor systems , 2014, Proceedings of the National Academy of Sciences.

[44]  E. Knudsen The Hearing of the Barn Owl , 1981 .

[45]  N. I. Fisher,et al.  Statistical Analysis of Circular Data , 1993 .

[46]  Stephen V. David,et al.  Cortical Membrane Potential Signature of Optimal States for Sensory Signal Detection , 2015, Neuron.

[47]  Stefan Koelsch,et al.  Processing of hierarchical syntactic structure in music , 2013, Proceedings of the National Academy of Sciences.

[48]  J. Stern,et al.  The endogenous eyeblink. , 1984, Psychophysiology.

[49]  S. Furukawa,et al.  Human Pupillary Dilation Response to Deviant Auditory Stimuli: Effects of Stimulus Properties and Voluntary Attention , 2016, Front. Neurosci..

[50]  Martin Cooke,et al.  A glimpsing model of speech perception in noise. , 2006, The Journal of the Acoustical Society of America.

[51]  Sylvain Baillet,et al.  Motor origin of temporal predictions in auditory attention , 2017, Proceedings of the National Academy of Sciences.

[52]  J. Coull fMRI studies of temporal attention: allocating attention within, or towards, time. , 2004, Brain research. Cognitive brain research.

[53]  David Poeppel,et al.  The coupling between auditory and motor cortices is rate-restricted: Evidence for an intrinsic speech-motor rhythm , 2018, Science Advances.

[54]  Wen Zhang,et al.  Attention Is Required for Knowledge-Based Sequential Grouping: Insights from the Integration of Syllables into Words , 2017, The Journal of Neuroscience.

[55]  K. Fukuda,et al.  Analysis of Eyeblink Activity during Discriminative Tasks , 1994, Perceptual and motor skills.

[56]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[57]  Edward W. Large,et al.  Perceiving temporal regularity in music , 2002, Cogn. Sci..

[58]  Lars Meyer,et al.  Linguistic Bias Modulates Interpretation of Speech via Neural Delta-Band Oscillations , 2016, Cerebral cortex.

[59]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[60]  G. Karmos,et al.  Entrainment of Neuronal Oscillations as a Mechanism of Attentional Selection , 2008, Science.

[61]  Jeroen van den Brink,et al.  The quantum nature of skyrmions and half-skyrmions in Cu2OSeO3 , 2014, Nature Communications.

[62]  M. Mauk,et al.  Extinction of conditioned eyelid responses requires the anterior lobe of cerebellar cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[63]  A. Meyer,et al.  Using the visual world paradigm to study language processing: a review and critical evaluation. , 2011, Acta psychologica.

[64]  Justin London,et al.  Book Review: Hearing in Time: Psychological Aspects of Musical Meter (2nd Edition) , 2004 .

[65]  M. Turvey,et al.  The motor theory of speech perception reviewed , 2006, Psychonomic bulletin & review.

[66]  M. Corbetta,et al.  A Common Network of Functional Areas for Attention and Eye Movements , 1998, Neuron.

[67]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[68]  C. Palmer Music performance. , 1997, Annual review of psychology.

[69]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[70]  Lisa D. Sanders,et al.  Listeners modulate temporally selective attention during natural speech processing , 2009, Biological Psychology.

[71]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[72]  Hugo Merchant,et al.  Finding the beat: a neural perspective across humans and non-human primates , 2015, Philosophical Transactions of the Royal Society B: Biological Sciences.

[73]  G. Fitzgerald,et al.  'I. , 2019, Australian journal of primary health.

[74]  L. Fadiga,et al.  Active perception: sensorimotor circuits as a cortical basis for language , 2010, Nature Reviews Neuroscience.

[75]  Joachim Gross,et al.  Perceptually relevant speech tracking in auditory and motor cortex reflects distinct linguistic features , 2018, PLoS biology.

[76]  Nicholas I. Fisher,et al.  Statistical Analysis of Circular Data , 1993 .

[77]  J. Fodor,et al.  Semantic focus and sentence comprehension , 1979, Cognition.

[78]  Kara D. Federmeier,et al.  Electrophysiology reveals semantic memory use in language comprehension , 2000, Trends in Cognitive Sciences.

[79]  J. Obleser,et al.  Frequency modulation entrains slow neural oscillations and optimizes human listening behavior , 2012, Proceedings of the National Academy of Sciences.

[80]  M. Murray,et al.  Multisensory Integration: Flexible Use of General Operations , 2014, Neuron.