Low-Frequency Oscillations Code Speech during Verbal Working Memory

The way the human brain represents speech in memory is still unknown. An obvious characteristic of speech is its evolvement over time. During speech processing, neural oscillations are modulated by the temporal properties of the acoustic speech signal, but also acquired knowledge on the temporal structure of language influences speech perception-related brain activity. This suggests that speech could be represented in the temporal domain, a form of representation that the brain also uses to encode autobiographic memories. Empirical evidence for such a memory code is lacking. We investigated the nature of speech memory representations using direct cortical recordings in the left perisylvian cortex during delayed sentence reproduction in female and male patients undergoing awake tumor surgery. Our results reveal that the brain endogenously represents speech in the temporal domain. Temporal pattern similarity analyses revealed that the phase of frontotemporal low-frequency oscillations, primarily in the beta range, represents sentence identity in working memory. The positive relationship between beta power during working memory and task performance suggests that working memory representations benefit from increased phase separation. SIGNIFICANCE STATEMENT Memory is an endogenous source of information based on experience. While neural oscillations encode autobiographic memories in the temporal domain, little is known on their contribution to memory representations of human speech. Our electrocortical recordings in participants who maintain sentences in memory identify the phase of left frontotemporal beta oscillations as the most prominent information carrier of sentence identity. These observations provide evidence for a theoretical model on speech memory representations and explain why interfering with beta oscillations in the left inferior frontal cortex diminishes verbal working memory capacity. The lack of sentence identity coding at the syllabic rate suggests that sentences are represented in memory in a more abstract form compared with speech coding during speech perception and production.

[1]  S. Hanslmayr,et al.  Temporal-Pattern Similarity Analysis Reveals the Beneficial and Detrimental Effects of Context Reinstatement on Human Memory , 2015, The Journal of Neuroscience.

[2]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[3]  N. Burgess,et al.  Brain oscillations and memory , 2010, Current Opinion in Neurobiology.

[4]  H. Bowman,et al.  The Temporal Signature of Memories: Identification of a General Mechanism for Dynamic Memory Replay in Humans , 2016, PLoS biology.

[5]  David Poeppel,et al.  The effects of selective attention and speech acoustics on neural speech-tracking in a multi-talker scene , 2015, Cortex.

[6]  Bijan Pesaran,et al.  Manipulating stored phonological input during verbal working memory , 2016, Nature Neuroscience.

[7]  D. Poeppel,et al.  Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language , 2004, Cognition.

[8]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[9]  Craig G. Richter,et al.  Interareal oscillatory synchronization in top-down neocortical processing , 2015, Current Opinion in Neurobiology.

[10]  Liberty S. Hamilton,et al.  Intonational speech prosody encoding in the human auditory cortex , 2017, Science.

[11]  A. Baddeley Working memory and language: an overview. , 2003, Journal of communication disorders.

[12]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[13]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[14]  Oleg Korzyukov,et al.  Functional role of delta and theta band oscillations for auditory feedback processing during vocal pitch motor control , 2015, Front. Neurosci..

[15]  F. Varela,et al.  Measuring phase synchrony in brain signals , 1999, Human brain mapping.

[16]  Bradley R. Buchsbaum,et al.  The Search for the Phonological Store: From Loop to Convolution , 2008, Journal of Cognitive Neuroscience.

[17]  B. Ross,et al.  Internalized Timing of Isochronous Sounds Is Represented in Neuromagnetic Beta Oscillations , 2012, The Journal of Neuroscience.

[18]  S. Nagarajan,et al.  Parsing the Phonological Loop: Activation Timing in the Dorsal Speech Stream Determines Accuracy in Speech Reproduction , 2013, The Journal of Neuroscience.

[19]  M. Potter,et al.  Syntactic Priming in Immediate Recall of Sentences , 1998 .

[20]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[21]  N. Mesgarani,et al.  Selective cortical representation of attended speaker in multi-talker speech perception , 2012, Nature.

[22]  Earl K. Miller,et al.  Working Memory 2.0 , 2018, Neuron.

[23]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[24]  Edward F Chang,et al.  The auditory representation of speech sounds in human motor cortex , 2016, eLife.

[25]  Steven Greenberg,et al.  On the Possible Role of Brain Rhythms in Speech Perception: Intelligibility of Time-Compressed Speech with Periodic and Aperiodic Insertions of Silence , 2009, Phonetica.

[26]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[27]  U. Ziemann,et al.  Left dorsal speech stream components and their contribution to phonological processing , 2015, Brain Stimulation.

[28]  W. Singer,et al.  Visuomotor integration is associated with zero time-lag synchronization among cortical areas , 1997, Nature.

[29]  Earl K Miller,et al.  Gamma and beta bursts during working memory readout suggest roles in its volitional control , 2017, Nature Communications.

[30]  A. Graybiel,et al.  Synchronous, Focally Modulated β-Band Oscillations Characterize Local Field Potential Activity in the Striatum of Awake Behaving Monkeys , 2003, The Journal of Neuroscience.

[31]  Christopher K. Kovach,et al.  Temporal Envelope of Time-Compressed Speech Represented in the Human Auditory Cortex , 2009, The Journal of Neuroscience.

[32]  Frank H Guenther,et al.  The DIVA model: A neural theory of speech acquisition and production , 2011, Language and cognitive processes.

[33]  Earl K Miller,et al.  Working Memory: Delay Activity, Yes! Persistent Activity? Maybe Not , 2018, The Journal of Neuroscience.

[34]  David Poeppel,et al.  The Tracking of Speech Envelope in the Human Cortex , 2013, PloS one.

[35]  Oded Ghitza,et al.  Linking Speech Perception and Neurophysiology: Speech Decoding Guided by Cascaded Oscillators Locked to the Input Rhythm , 2011, Front. Psychology.

[36]  K. Zaghloul,et al.  Reinstatement of distributed cortical oscillations occurs with precise spatiotemporal dynamics during successful memory retrieval , 2014, Proceedings of the National Academy of Sciences.

[37]  Simon Hanslmayr,et al.  Brain Oscillations Mediate Successful Suppression of Unwanted Memories. , 2015, Cerebral cortex.

[38]  J. Simon,et al.  Cortical entrainment to continuous speech: functional roles and interpretations , 2014, Front. Hum. Neurosci..

[39]  Stanislas Dehaene,et al.  Neurophysiological dynamics of phrase-structure building during sentence processing , 2017, Proceedings of the National Academy of Sciences.

[40]  Xiao-Jing Wang Neurophysiological and computational principles of cortical rhythms in cognition. , 2010, Physiological reviews.

[41]  P. Schyns,et al.  Speech Rhythms and Multiplexed Oscillatory Sensory Coding in the Human Brain , 2013, PLoS biology.

[42]  R. Henson,et al.  Stimulus–response bindings in priming , 2014, Trends in Cognitive Sciences.

[43]  Alexander Borst,et al.  How does Nature Program Neuron Types? , 2008, Front. Neurosci..

[44]  MenY C. Porren Regeneration i the Short-Term Recall of Sentences , 2006 .

[45]  D. Louis Collins,et al.  Symmetric Atlasing and Model Based Segmentation: An Application to the Hippocampus in Older Adults , 2006, MICCAI.

[46]  H. Kennedy,et al.  Alpha-Beta and Gamma Rhythms Subserve Feedback and Feedforward Influences among Human Visual Cortical Areas , 2016, Neuron.

[47]  Bijan Pesaran,et al.  Sensory-motor transformations for speech occur bilaterally , 2014, Nature.

[48]  A. Graybiel,et al.  Habit learning is associated with major shifts in frequencies of oscillatory activity and synchronized spike firing in striatum , 2011, Proceedings of the National Academy of Sciences.

[49]  Rodrigo F. Salazar,et al.  Content-Specific Fronto-Parietal Synchronization During Visual Working Memory , 2012, Science.

[50]  Ciprian M. Crainiceanu,et al.  Dynamics of large-scale cortical interactions at high gamma frequencies during word production: Event related causality (ERC) analysis of human electrocorticography (ECoG) , 2011, NeuroImage.

[51]  Heidi E Kirsch,et al.  Single-Trial Speech Suppression of Auditory Cortex Activity in Humans , 2010, The Journal of Neuroscience.

[52]  Sophie K. Scott,et al.  What is the relationship between phonological short-term memory and speech processing? , 2006, Trends in Cognitive Sciences.

[53]  Manuel R. Mercier,et al.  Evaluation of cortical local field potential diffusion in stereotactic electro-encephalography recordings: A glimpse on white matter signal , 2017, NeuroImage.

[54]  Eric Leuthardt,et al.  Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition , 2011, NeuroImage.

[55]  Markus Siegel,et al.  Phase-dependent neuronal coding of objects in short-term memory , 2009, Proceedings of the National Academy of Sciences.

[56]  W. Levelt,et al.  The spatial and temporal signatures of word production components , 2004, Cognition.

[57]  Kristofer E. Bouchard,et al.  Functional Organization of Human Sensorimotor Cortex for Speech Articulation , 2013, Nature.

[58]  A. Pérez-Villalba Rhythms of the Brain, G. Buzsáki. Oxford University Press, Madison Avenue, New York (2006), Price: GB £42.00, p. 448, ISBN: 0-19-530106-4 , 2008 .

[59]  J. Lachaux,et al.  What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring , 2014, Behavioural Brain Research.

[60]  Jutta L. Mueller,et al.  Brain Signature of Working Memory for Sentence Structure: Enriched Encoding and Facilitated Maintenance , 2014, Journal of Cognitive Neuroscience.

[61]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[62]  J. González-Martínez,et al.  Effect of invasive EEG monitoring on cognitive outcome after left temporal lobe epilepsy surgery , 2015, Neurology.

[63]  Ben Alderson-Day,et al.  Inner Speech: Development, Cognitive Functions, Phenomenology, and Neurobiology , 2015, Psychological bulletin.

[64]  Tanja Schultz,et al.  Brain-to-text: decoding spoken phrases from phone representations in the brain , 2015, Front. Neurosci..

[65]  Luc H. Arnal Predicting “When” Using the Motor System’s Beta-Band Oscillations , 2012, Front. Hum. Neurosci..

[66]  Cuntai Guan,et al.  Electrocorticographic representations of segmental features in continuous speech , 2015, Front. Hum. Neurosci..

[67]  Miles A Whittington,et al.  Potential Mechanisms Underlying Intercortical Signal Regulation via Cholinergic Neuromodulators , 2015, The Journal of Neuroscience.

[68]  Margitta Seeck,et al.  Focal versus distributed temporal cortex activity for speech sound category assignment , 2017, Proceedings of the National Academy of Sciences.

[69]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[70]  S. Hanslmayr,et al.  Entrainment of Prefrontal Beta Oscillations Induces an Endogenous Echo and Impairs Memory Formation , 2014, Current Biology.

[71]  Brian N. Pasley,et al.  Reconstructing Speech from Human Auditory Cortex , 2012, PLoS biology.

[72]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[73]  Lucia Melloni,et al.  Brain Oscillations during Spoken Sentence Processing , 2012, Journal of Cognitive Neuroscience.

[74]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[75]  Jutta L. Mueller,et al.  Oscillatory EEG dynamics underlying automatic chunking during sentence processing , 2017, NeuroImage.

[76]  E. Miller,et al.  Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices , 2007, Science.

[77]  H. Laufs,et al.  Non-linear Relationship between BOLD Activation and Amplitude of Beta Oscillations in the Supplementary Motor Area during Rhythmic Finger Tapping and Internal Timing , 2017, Front. Hum. Neurosci..

[78]  Peter Hagoort,et al.  Syntactic Unification Operations Are Reflected in Oscillatory Dynamics during On-line Sentence Comprehension , 2010, Journal of Cognitive Neuroscience.

[79]  Peter Hagoort,et al.  Neural activity during sentence processing as reflected in theta, alpha, beta, and gamma oscillations , 2016, NeuroImage.

[80]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[81]  Greg Gibson,et al.  Rare and common variants: twenty arguments , 2012, Nature Reviews Genetics.