Task-dependent modulation of the visual sensory thalamus assists visual-speech recognition

Abstract The cerebral cortex modulates early sensory processing via feed‐back connections to sensory pathway nuclei. The functions of this top‐down modulation for human behavior are poorly understood. Here, we show that top‐down modulation of the visual sensory thalamus (the lateral geniculate body, LGN) is involved in visual‐speech recognition. In two independent functional magnetic resonance imaging (fMRI) studies, LGN response increased when participants processed fast‐varying features of articulatory movements required for visual‐speech recognition, as compared to temporally more stable features required for face identification with the same stimulus material. The LGN response during the visual‐speech task correlated positively with the visual‐speech recognition scores across participants. In addition, the task‐dependent modulation was present for speech movements and did not occur for control conditions involving non‐speech biological movements. In face‐to‐face communication, visual speech recognition is used to enhance or even enable understanding what is said. Speech recognition is commonly explained in frameworks focusing on cerebral cortex areas. Our findings suggest that task‐dependent modulation at subcortical sensory stages has an important role for communication: Together with similar findings in the auditory modality the findings imply that task‐dependent modulation of the sensory thalami is a general mechanism to optimize speech recognition.

[1]  K. Uğurbil,et al.  Retinotopic mapping of lateral geniculate nucleus in humans using functional magnetic resonance imaging. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[2]  A. Cowey,et al.  The overrepresentation of the fovea and adjacent retina in the striate cortex and dorsal lateral geniculate nucleus of the macaque monkey , 1996, Neuroscience.

[3]  W. Marslen-Wilson,et al.  Continuous uptake of acoustic cues in spoken word recognition , 1987, Perception & psychophysics.

[4]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[5]  M. Nicolelis,et al.  Immediate thalamic sensory plasticity depends on corticothalamic feedback. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[6]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[7]  Trichur Raman Vidyasagar,et al.  The responses of cells in macaque lateral geniculate nucleus to sinusoidal gratings. , 1983, The Journal of physiology.

[8]  A. Sillito,et al.  Looking back: corticothalamic feedback and early visual processing , 2006, Trends in Neurosciences.

[9]  G Mann,et al.  ON THE THALAMUS * , 1905, British medical journal.

[10]  Sabine Kastner,et al.  Neural correlates of binocular rivalry in the human lateral geniculate nucleus , 2005, Nature Neuroscience.

[11]  Matthew H. Davis,et al.  Prediction Errors but Not Sharpened Signals Simulate Multivoxel fMRI Patterns during Speech Perception , 2016, PLoS biology.

[12]  R. Guillery,et al.  Thalamic Relay Functions and Their Role in Corticocortical Communication Generalizations from the Visual System , 2002, Neuron.

[13]  R. Romo,et al.  Coding perceptual discrimination in the somatosensory thalamus , 2012, Proceedings of the National Academy of Sciences.

[14]  R. Lal,et al.  Gating of retinal transmission by afferent eye position and movement signals. , 1989, Science.

[15]  B. Fraysse,et al.  Evidence that cochlear-implanted deaf patients are better multisensory integrators , 2007, Proceedings of the National Academy of Sciences.

[16]  J. Driver,et al.  Sound-Induced Enhancement of Low-Intensity Vision: Multisensory Influences on Human Sensory-Specific Cortices and Thalamic Bodies Relate to Perceptual Enhancement of Visual Detection Sensitivity , 2010, The Journal of Neuroscience.

[17]  Frank Tong,et al.  Attention alters orientation processing in the human lateral geniculate nucleus , 2015, Nature Neuroscience.

[18]  Karl J. Friston,et al.  Statistical parametric mapping , 2013 .

[19]  D. Massaro,et al.  The temporal distribution of information in audiovisual spoken-word identification , 2010, Attention, perception & psychophysics.

[20]  A. M. Sillito,et al.  Orientation sensitive elements in the corticofugal influence on centre-surround interactions in the dorsal lateral geniculate nucleus , 1993, Experimental Brain Research.

[21]  Essa Yacoub,et al.  Processing of frequency and location in human subcortical auditory structures , 2015, Scientific Reports.

[22]  D. Simons,et al.  Motor modulation of afferent somatosensory circuits , 2008, Nature Neuroscience.

[23]  J. Malpeli,et al.  The representation of the visual field in the lateral geniculate nucleus of Macaca mulatta , 1975, The Journal of comparative neurology.

[24]  W. Usrey,et al.  Emerging views of corticothalamic function , 2008, Current Opinion in Neurobiology.

[25]  A. Friederici,et al.  Lateralization of auditory language functions: A dynamic dual pathway model , 2004, Brain and Language.

[26]  Y. Saalmann,et al.  Cognitive and Perceptual Functions of the Visual Thalamus , 2011, Neuron.

[27]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[28]  Xenophon Papademetris,et al.  More accurate Talairach coordinates for neuroimaging using non-linear registration , 2008, NeuroImage.

[29]  G. Studebaker A "rationalized" arcsine transform. , 1985, Journal of speech and hearing research.

[30]  George L. Gerstein,et al.  Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex , 1994, Nature.

[31]  Nobuo Suga,et al.  Multiparametric corticofugal modulation and plasticity in the auditory system , 2003, Nature Reviews Neuroscience.

[32]  M. Pinsk,et al.  The Anatomical and Functional Organization of the Human Visual Pulvinar , 2015, The Journal of Neuroscience.

[33]  R. Hess,et al.  Responses of the human visual cortex and LGN to achromatic and chromatic temporal modulations: an fMRI study. , 2010, Journal of vision.

[34]  Simona Temereanca,et al.  Functional Topography of Corticothalamic Feedback Enhances Thalamic Spatial Response Tuning in the Somatosensory Whisker/Barrel System , 2004, Neuron.

[35]  Karl J. Friston,et al.  Lateral geniculate activations can be detected using intersubject averaging and fMRI , 1997, Magnetic resonance in medicine.

[36]  Torsten Marquardt,et al.  Representation of interaural time delay in the human auditory midbrain , 2006, Nature Neuroscience.

[37]  M. Koenigs,et al.  Ventromedial prefrontal cortex mediates visual attention during facial emotion recognition. , 2014, Brain : a journal of neurology.

[38]  W. R. Webster,et al.  Medial geniculate body of the cat: organization and responses to tonal stimuli of neurons in ventral division. , 1972, Journal of neurophysiology.

[39]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[40]  M. Nicolelis,et al.  Feature article: the structure and function of dynamic cortical and thalamic receptive fields. , 2001, Cerebral cortex.

[41]  P. Hluštík,et al.  Effects of spatial smoothing on fMRI group inferences. , 2008, Magnetic resonance imaging.

[42]  N. Suga,et al.  Corticofugal modulation of frequency processing in bat auditory system , 1997, Nature.

[43]  L. Mullins Do you see , 1970 .

[44]  D. Pollen,et al.  Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. , 1985, The Journal of physiology.

[45]  Keith A Schneider,et al.  Subcortical Mechanisms of Feature-Based Attention , 2011, The Journal of Neuroscience.

[46]  Alfred Anwander,et al.  Direct Structural Connections between Voice- and Face-Recognition Areas , 2011, The Journal of Neuroscience.

[47]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[48]  M. Pinsk,et al.  Attention modulates responses in the human lateral geniculate nucleus , 2002, Nature Neuroscience.

[49]  David Poeppel,et al.  Towards a New Neurobiology of Language , 2012, The Journal of Neuroscience.

[50]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[51]  Y. De Ribaupierre,et al.  Functional organization of the medial division of the medial geniculate body of the cat: Tonotopic organization, spatial distribution of response properties and cortical connections , 1989, Hearing Research.

[52]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[53]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[54]  Wolfgang Christian,et al.  Open Source Physics , 2011, Science.

[55]  J. Zihl,et al.  Speechreading in the akinetopsic patient , 1997 .

[56]  G. Rees,et al.  Saccades Differentially Modulate Human LGN and V1 Responses in the Presence and Absence of Visual Stimulation , 2005, Current Biology.

[57]  A. Sillito,et al.  Corticothalamic feedback enhances stimulus response precision in the visual system , 2007, Proceedings of the National Academy of Sciences.

[58]  Marlene C. Richter,et al.  Retinotopic Organization and Functional Subdivisions of the Human Lateral Geniculate Nucleus: A High-Resolution Functional Magnetic Resonance Imaging Study , 2004, The Journal of Neuroscience.

[59]  Farran Briggs,et al.  A Fast, Reciprocal Pathway between the Lateral Geniculate Nucleus and Visual Cortex in the Macaque Monkey , 2007, The Journal of Neuroscience.

[60]  L. Bernstein,et al.  Speech perception without hearing , 2000, Perception & psychophysics.

[61]  Sabine Kastner,et al.  Functional imaging of the human lateral geniculate nucleus and pulvinar. , 2004, Journal of neurophysiology.

[62]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[63]  D. Purves,et al.  Correlated Size Variations in Human Visual Cortex, Lateral Geniculate Nucleus, and Optic Tract , 1997, The Journal of Neuroscience.

[64]  P. C. Murphy,et al.  Corticofugal feedback influences the generation of length tuning in the visual pathway , 1987, Nature.

[65]  Deborah E. Hannula,et al.  The Eyes Have It: Hippocampal Activity Predicts Expression of Memory in Eye Movements , 2009, Neuron.

[66]  Sabine Kastner,et al.  Effects of Sustained Spatial Attention in the Human Lateral Geniculate Nucleus and Superior Colliculus , 2009, The Journal of Neuroscience.

[67]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[68]  Richard S. J. Frackowiak,et al.  Representation of the temporal envelope of sounds in the human brain. , 2000, Journal of neurophysiology.

[69]  S. Kiebel,et al.  Dysfunction of the auditory thalamus in developmental dyslexia , 2012, Proceedings of the National Academy of Sciences.

[70]  J. Winer The human medial geniculate body , 1984, Hearing Research.

[71]  M M Cohen,et al.  Speechreading in the akinetopsic patient, L.M. , 1997, Brain : a journal of neurology.

[72]  Katharina von Kriegstein,et al.  Mechanisms of enhancing visual–speech recognition by prior auditory information , 2013, NeuroImage.

[73]  J. Navarra,et al.  Hearing lips in a second language: visual articulatory information enables the perception of second language sounds , 2007, Psychological research.

[74]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[75]  J. Huguenard,et al.  Attentional flexibility in the thalamus: now we're getting SOMwhere , 2014, Nature Neuroscience.

[76]  P. Arnold,et al.  Bisensory augmentation: a speechreading advantage when speech is clearly audible and intact. , 2001, British journal of psychology.

[77]  R. Patterson,et al.  Task-Dependent Modulation of Medial Geniculate Body Is Behaviorally Relevant for Speech Recognition , 2008, Current Biology.

[78]  Richard S. J. Frackowiak,et al.  Cross-Modal Plasticity Underpins Language Recovery after Cochlear Implantation , 2001, Neuron.

[79]  S. R. Lehky,et al.  Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role , 2017, Progress in Neurobiology.

[80]  A. Sillito,et al.  Spatial frequency tuning of orientation‐discontinuity‐sensitive corticofugal feedback to the cat lateral geniculate nucleus. , 1996, The Journal of physiology.

[81]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[82]  Karl J. Friston,et al.  To Smooth or Not to Smooth? Bias and Efficiency in fMRI Time-Series Analysis , 2000, NeuroImage.

[83]  Karl J. Friston,et al.  Stochastic Designs in Event-Related fMRI , 1999, NeuroImage.