A sensorimotor view of verbal working memory

The divide-and-conquer approach to the study of human cognition has succeeded in focusing researchers' efforts on behavioral phenomena that fall under well-defined categories such as attention, perception, language, memory, emotion, and motor control. The result has been the development of coherent bodies of work in each area replete with successful explanatory theories and a rich collection of paradigms, tasks, and analytic techniques. There has been a renewed in recent years in combining and integrating ideas across these domains, as well as in incorporating neuroscientific data, as a way to build more powerful and general models of cognition. Here we review the history and current state of integration between verbal short-term memory (VeSTM) and language, two domains of study that have significant areas of overlap but have not been fully integrated. We review evidence from cognitive neuroscience that has generally shown VeSTM to greatly depend on the network of brain regions that are known to form the core sensory-motor basis of human language. Whereas classic psychological models of VeSTM posit the existence of dedicated short-term storage buffers, we suggest that temporary verbal memory emerges from the coordinated interplay of a fronto-temporal sensory-motor circuit that has evolved to support the perception and production of speech. Phonological rehearsal in the service of temporary maintenance is achieved by feedforward and feedback pathways connecting the auditory- and motor-speech systems via a sensorymotor interface component situated in the Sylvian-parietal-temporal region (Spt). Reciprocal connectivity between the frontal and temporal speech systems enables the kind of "round-tripping" of dual speech codes long hypothesized by cognitive models such as Baddeley and Hitch's "phonological loop".

[1]  A R Damasio,et al.  The anatomical basis of conduction aphasia. , 1980, Brain : a journal of neurology.

[2]  Gordon D. A. Brown,et al.  Serial Control of Phonology in Speech Production: A Hierarchical Model , 2000, Cognitive Psychology.

[3]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[4]  Sophie K. Scott,et al.  What is the relationship between phonological short-term memory and speech processing? , 2006, Trends in Cognitive Sciences.

[5]  U Klose,et al.  Dynamic pattern of brain activation during sequencing of word strings evaluated by fMRI. , 1999, Brain research. Cognitive brain research.

[6]  L. Shuster,et al.  An fMRI investigation of covertly and overtly produced mono- and multisyllabic words , 2005, Brain and Language.

[7]  Edward E. Smith,et al.  The Role of Parietal Cortex in Verbal Working Memory , 1998, The Journal of Neuroscience.

[8]  F. Guenther,et al.  A theoretical investigation of reference frames for the planning of speech movements. , 1998 .

[9]  B. Postle Working memory as an emergent property of the mind and brain , 2006, Neuroscience.

[10]  K. Berman,et al.  Human Dorsal and Ventral Auditory Streams Subserve Rehearsal-Based and Echoic Processes during Verbal Working Memory , 2005, Neuron.

[11]  M. Tsodyks,et al.  Working models of working memory , 2014, Current Opinion in Neurobiology.

[12]  Matthew M Botvinick,et al.  Short-term memory for serial order: a recurrent neural network model. , 2006, Psychological review.

[13]  C. Larson,et al.  Voice F0 responses to pitch-shifted auditory feedback: a preliminary study. , 1997, Journal of voice : official journal of the Voice Foundation.

[14]  J. Kalinowski,et al.  Effect of delayed auditory feedback on normal speakers at two speech rates. , 2002, The Journal of the Acoustical Society of America.

[15]  Tim Shallice,et al.  Auditory-verbal short-term memory impairment and conduction aphasia , 1977, Brain and Language.

[16]  M. L. Lambon Ralph,et al.  The roles of long-term phonotactic and lexical prosodic knowledge in phonological short-term memory , 2015, Memory & cognition.

[17]  Stefan Klöppel,et al.  Damage to ventral and dorsal language pathways in acute aphasia. , 2013 .

[18]  U. Ziemann,et al.  Left dorsal speech stream components and their contribution to phonological processing , 2015, Brain Stimulation.

[19]  Dennis Norris,et al.  Speech errors and the phonological similarity effect in short-term memory: Evidence suggesting a common locus , 2007 .

[20]  H. Seo,et al.  A reservoir of time constants for memory traces in cortical neurons , 2011, Nature Neuroscience.

[21]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.

[22]  Michael I. Jordan,et al.  Sensorimotor adaptation in speech production. , 1998, Science.

[23]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[24]  Daniel J. Acheson,et al.  Verbal working memory and language production: Common approaches to the serial ordering of verbal information. , 2009, Psychological bulletin.

[25]  D. Hebb Concerning imagery. , 1968, Psychological review.

[26]  F. Craik,et al.  Levels of Pro-cessing: A Framework for Memory Research , 1975 .

[27]  B. Postle,et al.  The cognitive neuroscience of working memory. , 2007, Annual review of psychology.

[28]  D. Hebb Textbook of psychology , 1958 .

[29]  Maryellen C. MacDonald,et al.  Speak, Act, Remember , 2016 .

[30]  D. Mackay,et al.  CONSTRAINTS ON THEORIES OF INNER SPEECH , 2014 .

[31]  Randi C. Martin,et al.  Language Processing and Working Memory: Neuropsychological Evidence for Separate Phonological and Semantic Capacities , 1994 .

[32]  P. A. Kolers,et al.  Procedures of mind. , 1984 .

[33]  H. Roediger,et al.  Memory metaphors in cognitive psychology , 1980, Memory & cognition.

[34]  Thomas J. Grabowski,et al.  The Left Posterior Superior Temporal Gyrus Participates Specifically in Accessing Lexical Phonology , 2008, Journal of Cognitive Neuroscience.

[35]  Margaret Wilson The case for sensorimotor coding in working memory , 2001, Psychonomic bulletin & review.

[36]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[37]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[38]  Edward F Chang,et al.  The cortical computations underlying feedback control in vocal production , 2015, Current Opinion in Neurobiology.

[39]  T. Whitford,et al.  Neurophysiological evidence of efference copies to inner speech , 2017, eLife.

[40]  M. Stokes ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework , 2015, Trends in Cognitive Sciences.

[41]  Cathy J. Price,et al.  Predicting language outcome and recovery after stroke: the PLORAS system , 2010, Nature Reviews Neurology.

[42]  Dennis Norris,et al.  Short-Term Memory and Long-Term Memory are Still Different , 2017, Psychological bulletin.

[43]  Kayoko Okada,et al.  Two cortical mechanisms support the integration of visual and auditory speech: A hypothesis and preliminary data , 2009, Neuroscience Letters.

[44]  Daniel J. Acheson,et al.  Areas of left perisylvian cortex mediate auditory–verbal short-term memory , 2011, Neuropsychologia.

[45]  Steven A. Jax,et al.  The problem of serial order in behavior: Lashley's legacy. , 2007, Human movement science.

[46]  T. Hackett,et al.  Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys , 2007, The Journal of comparative neurology.

[47]  S. Nagarajan,et al.  Parsing the Phonological Loop: Activation Timing in the Dorsal Speech Stream Determines Accuracy in Speech Reproduction , 2013, The Journal of Neuroscience.

[48]  Adam C. Riggall,et al.  Reactivation of latent working memories with transcranial magnetic stimulation , 2016, Science.

[49]  Olga Livanis What Was I Thinking , 2006 .

[50]  Flavio Dell'Acqua,et al.  Anatomical predictors of aphasia recovery: a tractography study of bilateral perisylvian language networks. , 2014, Brain : a journal of neurology.

[51]  Anne Cutler,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[52]  Colin Humphries,et al.  Role of left posterior superior temporal gyrus in phonological processing for speech perception and production , 2001 .

[53]  Judy Pa,et al.  A parietal–temporal sensory–motor integration area for the human vocal tract: Evidence from an fMRI study of skilled musicians , 2008, Neuropsychologia.

[54]  Nick C. Ellis,et al.  Memory for language , 2001 .

[55]  W. Kessen A Textbook of Psychology , 1958, The Yale Journal of Biology and Medicine.

[56]  Gary S Dell,et al.  Naming and repetition in aphasia: Steps, routes, and frequency effects. , 2010, Journal of memory and language.

[57]  G. Hickok,et al.  Auditory–Motor Interaction Revealed by fMRI: Speech, Music, and Working Memory in Area Spt , 2003 .

[58]  H. Axer,et al.  Supra- and Infrasylvian Conduction Aphasia , 2001, Brain and Language.

[59]  G. Vallar The neuropsychology of human memory , 2005, Neurocase.

[60]  Nina F. Dronkers,et al.  It’s either a cook or a baker: Patients with conduction aphasia get the gist but lose the trace , 2008, Brain and Language.

[61]  Robert G. Crowder,et al.  Short-term memory: Where do we stand? , 1993, Memory & cognition.

[62]  J. Fiez,et al.  A Comment on the Functional Localization of the Phonological Storage Subsystem of Working Memory , 1999, Brain and Cognition.

[63]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[64]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[65]  Alberto Bizzi,et al.  Disentangling subgroups of participants recruiting shared as well as different brain regions for the execution of the verb generation task: A data-driven fMRI study , 2017, Cortex.

[66]  Cathy J. Price,et al.  Inter-subject variability in the use of two different neuronal networks for reading aloud familiar words , 2008, NeuroImage.

[67]  Bradley R. Buchsbaum,et al.  The Search for the Phonological Store: From Loop to Convolution , 2008, Journal of Cognitive Neuroscience.

[68]  Prahlad Gupta,et al.  Word learning, phonological short-term memory, phonotactic probability and long-term memory: towards an integrated framework , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[69]  Christopher J. Markiewicz,et al.  Mapping the cortical representation of speech sounds in a syllable repetition task , 2016, NeuroImage.

[70]  Yaoda Xu Reevaluating the Sensory Account of Visual Working Memory Storage , 2017, Trends in Cognitive Sciences.

[71]  Richard L. Lewis,et al.  The mind and brain of short-term memory. , 2008, Annual review of psychology.

[72]  G. Waters,et al.  Aphasic disorders of syntactic comprehension and working memory capacity , 1995 .

[74]  Russell A. Poldrack,et al.  The Interface Between Neuroscience and Psychological Science , 2008 .

[75]  Gary M. Oppenheim,et al.  Motor movement matters: The flexible abstractness of inner speech , 2010, Memory & cognition.

[76]  Andrew W. Ellis,et al.  Errors in speech and short-term memory: The effects of phonemic similarity and syllable position , 1980 .

[77]  D. Poeppel A Critical Review of PET Studies of Phonological Processing , 1996, Brain and Language.

[78]  J. Lachaux,et al.  What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring , 2014, Behavioural Brain Research.

[79]  Argye E Hillis,et al.  Recovery from aphasia following brain injury: the role of reorganization. , 2006, Progress in brain research.

[80]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[81]  Milene Bonte,et al.  Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions , 2015, The Journal of Neuroscience.

[82]  Xiao-Jing Wang Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[83]  D. Howes,et al.  The Nature of Conduction Aphasia: A Study of Anatomic and Clinical Features and of Underlying Mechanisms , 1977 .

[84]  D. Caplan,et al.  A case study of reproduction conduction aphasia I: Word production , 1986 .

[85]  Ivan Toni,et al.  Task instructions influence the cognitive strategies involved in line bisection judgements: evidence from modulated neural mechanisms revealed by fMRI , 2002, Neuropsychologia.

[86]  Kayoko Okada,et al.  Conduction aphasia, sensory-motor integration, and phonological short-term memory – An aggregate analysis of lesion and fMRI data , 2011, Brain and Language.

[87]  A. Baddeley,et al.  Word length and the structure of short-term memory , 1975 .

[88]  C. Degueldre,et al.  Regional brain activity during working memory tasks. , 1996, Brain : a journal of neurology.

[89]  Tim Shallice,et al.  Neuropsychological impairments of short-term memory , 1992 .

[90]  A. Baddeley The episodic buffer: a new component of working memory? , 2000, Trends in Cognitive Sciences.

[91]  Tim Shallice,et al.  Neuropsychological Impairments of Short-Term Memory , 1992 .

[92]  N. Geschwind,et al.  Conduction aphasia. A clinicopathological study. , 1973, Archives of neurology.

[93]  Mark D'Esposito,et al.  From cognitive to neural models of working memory , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[94]  Richard S. J. Frackowiak,et al.  Anatomical Modularity of Verbal Working Memory? Functional Anatomical Evidence from a Famous Patient with Short-Term Memory Deficits , 2017, Front. Hum. Neurosci..

[95]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[96]  A. Galaburda,et al.  Cytoarchitectonic organization of the human auditory cortex , 1980, The Journal of comparative neurology.

[97]  Donald G. MacKay,et al.  H.M. Revisited: Relations between Language Comprehension, Memory, and the Hippocampal System , 1998, Journal of Cognitive Neuroscience.

[98]  Alan D Baddeley,et al.  Memory for serial order across domains: An overview of the literature and directions for future research. , 2014, Psychological bulletin.

[99]  Stephen E. Nadeau,et al.  Phonology: A Review and Proposals from a Connectionist Perspective , 2001, Brain and Language.

[100]  R. Andersen,et al.  The posterior parietal cortex: Sensorimotor interface for the planning and online control of visually guided movements , 2006, Neuropsychologia.

[101]  Randi C. Martin,et al.  Dissociations between speech perception and phonological short-term memory deficits , 1992 .

[102]  Alan D. Baddeley,et al.  Disruption of short-term memory by unattended speech : Implications for the structure of working memory , 1982 .

[103]  T Shallice,et al.  The selective impairment of auditory verbal short-term memory. , 1969, Brain : a journal of neurology.

[104]  D. G. MacKay Aspects of the Theory of Comprehension, Memory and Attention , 1973 .

[105]  A. Baddeley,et al.  When long-term learning depends on short-term storage , 1988 .

[106]  Jesper Andersson,et al.  A multi-modal parcellation of human cerebral cortex , 2016, Nature.

[107]  Edward E. Smith,et al.  Verbal Working Memory Load Affects Regional Brain Activation as Measured by PET , 1997, Journal of Cognitive Neuroscience.

[108]  Harold Goodglass,et al.  Diagnosis of Conduction Aphasia , 2013 .

[109]  Bijan Pesaran,et al.  Manipulating stored phonological input during verbal working memory , 2016, Nature Neuroscience.

[110]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[111]  Kayoko Okada,et al.  Neural evidence for predictive coding in auditory cortex during speech production , 2018, Psychonomic bulletin & review.

[112]  Qiuhai Yue,et al.  Non-perceptual Regions in the Left Inferior Parietal Lobe Support Phonological Short-term Memory: Evidence for a Buffer Account? , 2019, Cerebral cortex.

[113]  G. Dell,et al.  A Case-Series Test of the Interactive Two-step Model of Lexical Access: Predicting Word Repetition from Picture Naming. , 2007, Journal of memory and language.

[114]  C. Hulme,et al.  Speech and language processing mechanisms in verbal serial recall , 2006 .

[115]  Mark D'Esposito,et al.  The effect of rehearsal rate and memory load on verbal working memory , 2015, NeuroImage.

[116]  R. Schubotz Opinion TRENDS in Cognitive Sciences Vol.11 No.5 Prediction , 2022 .

[117]  Tim Shallice,et al.  Neuropsychological impairments of short-term memory: The impairment of auditory–verbal short-term storage , 1990 .

[118]  Daniel J. Acheson,et al.  A Common Neural Substrate for Language Production and Verbal Working Memory , 2011, Journal of Cognitive Neuroscience.

[119]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[120]  D. Schacter,et al.  The cognitive neuroscience of constructive memory: remembering the past and imagining the future , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[121]  Gregory Hickok,et al.  Towards an integrated psycholinguistic, neurolinguistic, sensorimotor framework for speech production , 2014, Language and cognitive processes.

[122]  Jessica S. Arsenault,et al.  Distributed Neural Representations of Phonological Features during Speech Perception , 2015, The Journal of Neuroscience.

[123]  D. Caplan,et al.  Short-term memory, working memory, and syntactic comprehension in aphasia , 2013, Cognitive neuropsychology.

[124]  N. Martin,et al.  Neuropsychological impairments of short-term memory: Neuropsychological evidence for lexical involvement in short-term memory , 1990 .

[125]  D. Norris,et al.  The primacy model: a new model of immediate serial recall. , 1998, Psychological review.

[126]  P. Pietrini,et al.  Functional and spatial segregation within the inferior frontal and superior temporal cortices during listening, articulation imagery, and production of vowels , 2017, Scientific Reports.

[127]  Nina F. Dronkers,et al.  The Pursuit of Brain–Language Relationships , 2000, Brain and Language.

[128]  Jean K. Gordon,et al.  A Neural Signature of Phonological Access: Distinguishing the Effects of Word Frequency from Familiarity and Length in Overt Picture Naming , 2007, Journal of Cognitive Neuroscience.

[129]  M. Scott Corollary Discharge Provides the Sensory Content of Inner Speech , 2013, Psychological science.

[130]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[131]  A. J. Yates Delayed auditory feedback. , 1963 .

[132]  D. Poeppel,et al.  Health, USA Reviewed by: , 2010 .

[133]  Bradley R. Buchsbaum,et al.  Reading, hearing, and the planum temporale , 2005, NeuroImage.

[134]  Maria Luisa Gorno-Tempini,et al.  Connected speech production in three variants of primary progressive aphasia. , 2010, Brain : a journal of neurology.

[135]  Gary S. Dell,et al.  Inner speech slips exhibit lexical bias, but not the phonemic similarity effect , 2008, Cognition.

[136]  Lyndsey Nickels,et al.  Separating input and output phonology: semantic, phonological, and orthographic effects in short-term memory impairment , 2005, Cognitive neuropsychology.

[137]  N. Burgess,et al.  Computational models of working memory: putting long-term memory into context , 2005, Trends in Cognitive Sciences.

[138]  Jessica S McQuiggan,et al.  What Was I Thinking? Using Multi-Voxel Pattern Analysis to Identify Shared Neural Representations Between External and Internal Speech , 2017 .

[139]  Richard S. J. Frackowiak,et al.  The neural correlates of the verbal component of working memory , 1993, Nature.

[140]  Chris Rorden,et al.  Sensorimotor impairment of speech auditory feedback processing in aphasia , 2018, NeuroImage.

[141]  Robert J. Hartsuiker,et al.  Error Monitoring in Speech Production: A Computational Test of the Perceptual Loop Theory , 2001, Cognitive Psychology.

[142]  R. Poldrack Can cognitive processes be inferred from neuroimaging data? , 2006, Trends in Cognitive Sciences.