Transformation of speech sequences in human sensorimotor circuits

After we listen to a series of words, we can silently replay them in our mind. Does this mental replay involve a re-activation of our original perceptual representations? We recorded electrocorticographic (ECoG) activity across the lateral cerebral cortex as people heard and then mentally rehearsed spoken sentences. For each region, we tested whether silent rehearsal of sentences involved reactivation of sentence-specific representations established during perception or transformation to a distinct representation. In sensorimotor and premotor cortex, we observed reliable and temporally precise responses to speech; these patterns transformed to distinct sentence-specific representations during mental rehearsal. In contrast, we observed slower and less reliable responses in prefrontal and temporoparietal cortex; these higher-order representations, which were sensitive to sentence semantics, were shared across perception and rehearsal. The mental rehearsal of natural speech involves the transformation of time-resolved speech representations in sensorimotor and premotor cortex, combined with diffuse reactivation of higher-order semantic representations. Conflict of interest The authors declare no competing financial interests.

[1]  L. Fadiga,et al.  The Motor Somatotopy of Speech Perception , 2009, Current Biology.

[2]  J. B. Jackson,et al.  "How do you know what you know?". , 2004, HIV prevention plus!.

[3]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[4]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[5]  Jutta L. Mueller,et al.  Brain Signature of Working Memory for Sentence Structure: Enriched Encoding and Facilitated Maintenance , 2014, Journal of Cognitive Neuroscience.

[6]  R. Tibshirani,et al.  The solution path of the generalized lasso , 2010, 1005.1971.

[7]  Nancy Kanwisher,et al.  Neural correlate of the construction of sentence meaning , 2016, Proceedings of the National Academy of Sciences.

[8]  David J. Freedman,et al.  A hierarchy of intrinsic timescales across primate cortex , 2014, Nature Neuroscience.

[9]  Mary C. Potter,et al.  The regeneration of syntax in short term memory , 1992 .

[10]  D. Heeger,et al.  Slow Cortical Dynamics and the Accumulation of Information over Long Timescales , 2012, Neuron.

[11]  Mark D'Esposito,et al.  The effect of rehearsal rate and memory load on verbal working memory , 2015, NeuroImage.

[12]  L. Parra,et al.  Human Neuroscience Original Research Article Correlated Components of Ongoing Eeg Point to Emotionally Laden Attention – a Possible Marker of Engagement? , 2022 .

[13]  Simon Hanslmayr,et al.  Low-Frequency Oscillations Code Speech during Verbal Working Memory , 2019, The Journal of Neuroscience.

[14]  B. Gordon,et al.  Induced electrocorticographic gamma activity during auditory perception , 2001, Clinical Neurophysiology.

[15]  Michael H Kohrman,et al.  ECoG gamma activity during a language task: differentiating expressive and receptive speech areas. , 2008, Brain : a journal of neurology.

[16]  C. Honey,et al.  Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story , 2011, The Journal of Neuroscience.

[17]  Bradley R. Buchsbaum,et al.  The Search for the Phonological Store: From Loop to Convolution , 2008, Journal of Cognitive Neuroscience.

[18]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[19]  Edward F Chang,et al.  The auditory representation of speech sounds in human motor cortex , 2016, eLife.

[20]  E. Miller,et al.  Intrinsic neuronal dynamics predict distinct functional roles during working memory , 2017, Nature Communications.

[21]  Bijan Pesaran,et al.  Sensory-motor transformations for speech occur bilaterally , 2014, Nature.

[22]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[23]  Andreas Schulze-Bonhage,et al.  Real-life speech production and perception have a shared premotor-cortical substrate , 2018, Scientific Reports.

[24]  Rajesh P. N. Rao,et al.  Dynamic Modulation of Local Population Activity by Rhythm Phase in Human Occipital Cortex During a Visual Search Task , 2010, Front. Hum. Neurosci..

[25]  Bijan Pesaran,et al.  Manipulating stored phonological input during verbal working memory , 2016, Nature Neuroscience.

[26]  M. Potter,et al.  Regeneration in the short-term recall of sentences , 1990 .

[27]  Colin Humphries,et al.  Time course of semantic processes during sentence comprehension: An fMRI study , 2007, NeuroImage.

[28]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[29]  J. Maunsell,et al.  Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex , 2011, PLoS biology.

[30]  MenY C. Porren Regeneration i the Short-Term Recall of Sentences , 2006 .

[31]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[32]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[33]  D. V. Cramon,et al.  Functional–anatomical concepts of human premotor cortex: evidence from fMRI and PET studies , 2003, NeuroImage.

[34]  Friedemann Pulvermüller,et al.  Motor cortex maps articulatory features of speech sounds , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Shula Chiat,et al.  Sentence repetition: what does the task measure? , 2015, International journal of language & communication disorders.

[36]  Heidi E Kirsch,et al.  Single-Trial Speech Suppression of Auditory Cortex Activity in Humans , 2010, The Journal of Neuroscience.

[37]  Romain Quentin,et al.  Differential Brain Mechanisms of Selection and Maintenance of Information during Working Memory , 2019, The Journal of Neuroscience.

[38]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.

[39]  Daniela J. Palombo,et al.  Memory integration in amnesia: Prior knowledge supports verbal short-term memory , 2015, Neuropsychologia.

[40]  Meny C. PorrBn The Regeneration of Syntax in Short Term Memory , 2006 .

[41]  Steven W Kennerley,et al.  Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex , 2017, Nature Communications.

[42]  Jed A. Meltzer,et al.  Semantic and phonological contributions to short-term repetition and long-term cued sentence recall , 2015, Memory & Cognition.

[43]  M. Stokes ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework , 2015, Trends in Cognitive Sciences.

[44]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[45]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[46]  Hervé Abdi,et al.  The Neural Basis of Vivid Memory Is Patterned on Perception , 2012, Journal of Cognitive Neuroscience.

[47]  Elizabeth Jefferies,et al.  Keeping it together: Semantic coherence stabilizes phonological sequences in short-term memory , 2017, Memory & cognition.

[48]  Tyler K. Perrachione,et al.  Common cortical architectures for phonological working memory identified in individual brains , 2019, NeuroImage.

[49]  B. Gordon,et al.  Induced electrocorticographic gamma activity during auditory perception. Brazier Award-winning article, 2001. , 2001, Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology.

[50]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[51]  Erik Edwards,et al.  Comparison of time-frequency responses and the event-related potential to auditory speech stimuli in human cortex. , 2009, Journal of neurophysiology.

[52]  Jeremy R. Manning,et al.  Broadband Shifts in Local Field Potential Power Spectra Are Correlated with Single-Neuron Spiking in Humans , 2009, The Journal of Neuroscience.

[53]  Josef P. Rauschecker,et al.  An expanded role for the dorsal auditory pathway in sensorimotor control and integration , 2011, Hearing Research.

[54]  H. Bowman,et al.  The Temporal Signature of Memories: Identification of a General Mechanism for Dynamic Memory Replay in Humans , 2016, PLoS biology.

[55]  Christopher J. Honey,et al.  iELVis: An open source MATLAB toolbox for localizing and visualizing human intracranial electrode data , 2016, Journal of Neuroscience Methods.

[56]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[57]  Greg Gibson,et al.  Rare and common variants: twenty arguments , 2012, Nature Reviews Genetics.

[58]  Wayne E Mackey,et al.  Oscillatory recurrent gated neural integrator circuits (ORGaNICs), a unifying theoretical framework for neural dynamics , 2019, Proceedings of the National Academy of Sciences.

[59]  Jarrod A. Lewis-Peacock,et al.  Temporary Activation of Long-Term Memory Supports Working Memory , 2008, The Journal of Neuroscience.

[60]  Christopher J. Honey,et al.  Does rhythmic entrainment represent a generalized mechanism for organizing computation in the brain? , 2012, Front. Comput. Neurosci..

[61]  Sean M. Polyn,et al.  A context maintenance and retrieval model of organizational processes in free recall. , 2009, Psychological review.

[62]  Anders M. Dale,et al.  Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature , 2010, NeuroImage.

[63]  Yinjuan Du,et al.  Noise differentially impacts phoneme representations in the auditory and speech motor systems , 2014, Proceedings of the National Academy of Sciences.

[64]  Sophie K. Scott,et al.  What is the relationship between phonological short-term memory and speech processing? , 2006, Trends in Cognitive Sciences.

[65]  Jessica S. Arsenault,et al.  No evidence of somatotopic place of articulation feature mapping in motor cortex during passive speech perception , 2016, Psychonomic bulletin & review.

[66]  Eric Leuthardt,et al.  Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition , 2011, NeuroImage.

[67]  R. Knight,et al.  Redefining the role of Broca’s area in speech , 2015, Proceedings of the National Academy of Sciences.