Manipulating stored phonological input during verbal working memory

Verbal working memory (vWM) involves storing and manipulating information in phonological sensory input. An influential theory of vWM proposes that manipulation is carried out by a central executive while storage is performed by two interacting systems: a phonological input buffer that captures sound-based information and an articulatory rehearsal system that controls speech motor output. Whether, when and how neural activity in the brain encodes these components remains unknown. Here we read out the contents of vWM from neural activity in human subjects as they manipulated stored speech sounds. As predicted, we identified storage systems that contained both phonological sensory and articulatory motor representations. Unexpectedly, however, we found that manipulation did not involve a single central executive but rather involved two systems with distinct contributions to successful manipulation. We propose, therefore, that multiple subsystems comprise the central executive needed to manipulate stored phonological input for articulatory motor output in vWM.

[1]  Emmanuel Dupoux,et al.  Misperception in sentences but not in words: Speech perception and the phonological buffer , 2006, Cognitive neuropsychology.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  G. Hickok,et al.  Auditory–Motor Interaction Revealed by fMRI: Speech, Music, and Working Memory in Area Spt , 2003 .

[4]  A. Baddeley,et al.  Exploring the Articulatory Loop , 1984 .

[5]  E. Niebur,et al.  Neural Correlates of High-Gamma Oscillations (60–200 Hz) in Macaque Local Field Potentials and Their Potential Implications in Electrocorticography , 2008, The Journal of Neuroscience.

[6]  I. Fried,et al.  Coupling between Neuronal Firing Rate, Gamma LFP, and BOLD fMRI Is Related to Interneuronal Correlations , 2007, Current Biology.

[7]  Bradley R. Buchsbaum,et al.  The Search for the Phonological Store: From Loop to Convolution , 2008, Journal of Cognitive Neuroscience.

[8]  B. Postle,et al.  Functional neuroanatomical double dissociation of mnemonic and executive control processes contributing to working memory performance. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Edward E. Smith,et al.  Verbal Working Memory Load Affects Regional Brain Activation as Measured by PET , 1997, Journal of Cognitive Neuroscience.

[10]  E. Vogel,et al.  Interactions between attention and working memory , 2006, Neuroscience.

[11]  Jonathan W. Pillow,et al.  Single-trial spike trains in parietal cortex reveal discrete steps during decision-making , 2015, Science.

[12]  R. Passingham,et al.  The prefrontal cortex: response selection or maintenance within working memory? , 2000, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[13]  K. C. Anderson,et al.  Single neurons in prefrontal cortex encode abstract rules , 2001, Nature.

[14]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[15]  Richard S. J. Frackowiak,et al.  The neural correlates of the verbal component of working memory , 1993, Nature.

[16]  C. Curtis,et al.  Multiple component networks support working memory in prefrontal cortex , 2015, Proceedings of the National Academy of Sciences.

[17]  Christopher D. Manning,et al.  Advances in natural language processing , 2015, Science.

[18]  Orrin Devinsky,et al.  Localization of dense intracranial electrode arrays using magnetic resonance imaging , 2012, NeuroImage.

[19]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[20]  P. Ladefoged.  A course in phonetics , 1975 .

[21]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[22]  B. MacWhinney,et al.  Vocabulary Acquisition and Verbal Short-Term Memory: Computational and Neural Bases , 1997, Brain and Language.

[23]  Michael D. Kopelman,et al.  The handbook of memory disorders , 1995 .

[24]  A. Baddeley.  The Influence of Acoustic and Semantic Similarity on Long-term Memory for Word Sequences , 1966, The Quarterly journal of experimental psychology.

[25]  A. Baddeley,et al.  Phonological short-term store, phonological processing and sentence comprehension: A neuropsychological case study , 1984 .

[26]  Tim Shallice,et al.  Fractionation of the supervisory system. , 2002 .

[27]  Y. Miyashita,et al.  Neural organization for the long-term memory of paired associates , 1991, Nature.

[28]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[29]  R. Logie.  Spatial and Visual Working Memory: A Mental Workspace , 2003 .

[30]  A. Baddeley.  Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[31]  A. Baddeley,et al.  Word length and the structure of short-term memory , 1975 .

[32]  P. Mitra,et al.  Analysis of dynamic brain imaging data. , 1998, Biophysical journal.

[33]  Mingsha Zhang,et al.  Neuronal switching of sensorimotor transformations for antisaccades , 2000, Nature.

[34]  Sophie K. Scott,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.11 What , 2022 .

[35]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[36]  A. Baddeley,et al.  Phonological short-term memory and foreign-language vocabulary learning☆ , 1991 .

[37]  C. Curtis,et al.  Common neural mechanisms supporting spatial working memory, attention and motor intention , 2011, Neuropsychologia.

[38]  Bijan Pesaran,et al.  Sensory-motor transformations for speech occur bilaterally , 2014, Nature.

[39]  E. Chang,et al.  UC San Francisco UC San Francisco Previously Published Works Title Oscillatory dynamics coordinating human frontal networks in support of goal maintenance , 2015 .

[40]  N. Martin,et al.  Neuropsychological impairments of short-term memory: Short-term memory impairment and sentence processing: a case study , 1990 .

[41]  Y. Miyashita,et al.  Top-down signal from prefrontal cortex in executive control of memory retrieval , 1999, Nature.

[42]  Christopher D. Harvey,et al.  Choice-specific sequences in parietal cortex during a virtual-navigation decision task , 2012, Nature.

[43]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[44]  Raymond J Dolan,et al.  Maintenance versus manipulation in verbal working memory revisited: an fMRI study , 2003, NeuroImage.

[45]  M. D’Esposito,et al.  The neural basis of the central executive system of working memory , 1995, Nature.

[46]  D. Poeppel,et al.  Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language , 2004, Cognition.

[47]  P. Goldman-Rakic.  The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[48]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[49]  J. Fiez,et al.  A Comment on the Functional Localization of the Phonological Storage Subsystem of Working Memory , 1999, Brain and Cognition.

[50]  Colin Humphries,et al.  Role of left posterior superior temporal gyrus in phonological processing for speech perception and production , 2001, Cogn. Sci..

[51]  Giuseppe Vallar,et al.  Fractionation of working memory: Neuropsychological evidence for a phonological short-term store , 1984 .