Encoding, rehearsal, and recall in signers and speakers: shared network but differential engagement.

Short-term memory (STM), or the ability to hold verbal information in mind for a few seconds, is known to rely on the integrity of a frontoparietal network of areas. Here, we used functional magnetic resonance imaging to ask whether a similar network is engaged when verbal information is conveyed through a visuospatial language, American Sign Language, rather than speech. Deaf native signers and hearing native English speakers performed a verbal recall task, where they had to first encode a list of letters in memory, maintain it for a few seconds, and finally recall it in the order presented. The frontoparietal network described to mediate STM in speakers was also observed in signers, with its recruitment appearing independent of the modality of the language. This finding supports the view that signed and spoken STM rely on similar mechanisms. However, deaf signers and hearing speakers differentially engaged key structures of the frontoparietal network as the stages of STM unfold. In particular, deaf signers relied to a greater extent than hearing speakers on passive memory storage areas during encoding and maintenance, but on executive process areas during recall. This work opens new avenues for understanding similarities and differences in STM performance in signers and speakers.

[1]  Margaret Wilson,et al.  A visuospatial “phonological loop” in working memory: Evidence from American Sign Language , 1997, Memory & cognition.

[2]  T. Shallice,et al.  Short-term memory impairment and spontaneous speech , 1977, Neuropsychologia.

[3]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[4]  Mark D'Esposito,et al.  A subsequent-memory effect in dorsolateral prefrontal cortex. , 2003, Brain research. Cognitive brain research.

[5]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[6]  Edward E. Smith,et al.  Working Memory: A View from Neuroimaging , 1997, Cognitive Psychology.

[7]  A. Miyake,et al.  Models of Working Memory: Toward Unified Theories of Working Memory: Emerging General Consensus, Unresolved Theoretical Issues, and Future Research Directions , 1999 .

[8]  T Shallice,et al.  The dissociation between short term retention of meaningful sounds and verbal material. , 1974, Neuropsychologia.

[9]  Gregory G. Brown,et al.  Reproducibility of functional MR imaging: preliminary results of prospective multi-institutional study performed by Biomedical Informatics Research Network. , 2005, Radiology.

[10]  Julie A Fiez,et al.  Functional dissociations within the inferior parietal cortex in verbal working memory , 2004, NeuroImage.

[11]  Margaret Wilson,et al.  The effect of irrelevant visual input on working memory for sign language. , 2003, Journal of deaf studies and deaf education.

[12]  Margaret Wilson,et al.  Comparing Sign Language and Speech Reveals a Universal Limit on Short-Term Memory Capacity , 2006, Psychological science.

[13]  A. Miyake,et al.  Models of Working Memory: Mechanisms of Active Maintenance and Executive Control , 1999 .

[14]  Cecile McKee The Signs of Language Revisited: An Anthology to Honor Ursula Bellugi and Edward Klima (review) , 2001 .

[15]  N Burgess,et al.  Recoding, storage, rehearsal and grouping in verbal short-term memory: an fMRI study , 2000, Neuropsychologia.

[16]  Ursula Bellugi,et al.  Remembering in signs , 1975, Cognition.

[17]  Karen Emmorey,et al.  The neural correlates of sign versus word production , 2007, NeuroImage.

[18]  E. Klima The signs of language , 1979 .

[19]  B. Woll Review of “The Signs of Language Revisited: an Anthology to Honor Ursula Bellugi and Edward Klima” by Karen Emmorey & Harlan Lane (eds.) , 2002 .

[20]  R Conrad,et al.  Short-term memory processes in the deaf. , 1970, British journal of psychology.

[21]  J. Jonides,et al.  Dissociating verbal and spatial working memory using PET. , 1996, Cerebral cortex.

[22]  Henrik Walter,et al.  Evidence for Quantitative Domain Dominance for Verbal and Spatial Working Memory in Frontal and Parietal Cortex , 2003, Cortex.

[23]  M. Silveri,et al.  The phonological short-term store-rehearsal system: Patterns of impairment and neural correlates , 1997, Neuropsychologia.

[24]  Rachel I. Mayberry,et al.  The long-lasting advantage of learning sign language in childhood: Another look at the critical period for language acquisition , 1991 .

[25]  Margaret Wilson,et al.  When does modality matter? Evidence from ASL on the nature of working memory. , 2000 .

[26]  T. Braver,et al.  The Role of Frontopolar Cortex in Subgoal Processing during Working Memory , 2002, NeuroImage.

[27]  Jonathan D. Cohen,et al.  Reproducibility of fMRI Results across Four Institutions Using a Spatial Working Memory Task , 1998, NeuroImage.

[28]  Margaret Wilson,et al.  A “word length effect”for sign language: Further evidence for the role of language in structuring working memory , 1998, Memory & cognition.

[29]  Matthew L. Hall,et al.  Persistent Difference in Short-Term Memory Span Between Sign and Speech , 2006, Psychological science.

[30]  R Conrad,et al.  Short-term memory in the deaf: a test for speech coding. , 1972, British journal of psychology.

[31]  Beth L. Losiewicz A Specialized Language System in Working Memory: Evidence from American Sign Language. , 1999 .

[32]  V Menon,et al.  Modality effects in verbal working memory: differential prefrontal and parietal responses to auditory and visual stimuli , 2004, NeuroImage.

[33]  V. L. Hanson,et al.  Short-term recall by deaf signers of American Sign Language: implications of encoding strategy for order recall. , 1982, Journal of experimental psychology. Learning, memory, and cognition.

[34]  Edward E. Smith,et al.  PET Evidence for an Amodal Verbal Working Memory System , 1996, NeuroImage.

[35]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[36]  Yaakov Stern,et al.  An event-related fMRI study of the neural networks underlying the encoding, maintenance, and retrieval phase in a delayed-match-to-sample task. , 2005, Brain research. Cognitive brain research.

[37]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[38]  U. Bellugi,et al.  A comparison of sign language and spoken language , 1972 .

[39]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[40]  Jerker Rönnberg,et al.  Neural correlates of working memory for sign language. , 2004, Brain research. Cognitive brain research.

[41]  Elissa L. Newport,et al.  Ordered short-term memory differs in signers and speakers: Implications for models of short-term memory , 2008, Cognition.

[42]  R. Mayberry,et al.  First-language acquisition after childhood differs from second-language acquisition: the case of American Sign Language. , 1993, Journal of speech and hearing research.

[43]  R. Campbell,et al.  Varieties of short-term memory coding in deaf teenagers. , 1996, Journal of deaf studies and deaf education.

[44]  Russell A. Poldrack,et al.  Long-term test–retest reliability of functional MRI in a classification learning task , 2006, NeuroImage.

[45]  Mark S. Cohen,et al.  Parametric Analysis of fMRI Data Using Linear Systems Methods , 1997, NeuroImage.

[46]  Margaret Wilson,et al.  No Difference in Short-Term Memory Span Between Sign and Speech , 2006, Psychological science.

[47]  John D E Gabrieli,et al.  The role of the prefrontal cortex in the maintenance of verbal working memory: an event-related FMRI analysis. , 2005, Neuropsychology.

[48]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[49]  Daphne Bavelier,et al.  Short-term memory span: insights from sign language , 2004, Nature Neuroscience.

[50]  Gary H. Glover,et al.  Reducing inter-scanner variability of activation in a multicenter fMRI study: Role of smoothness equalization , 2006, NeuroImage.

[51]  Donald A. Norman,et al.  Attention to Action , 1986 .

[52]  M D'Esposito,et al.  The roles of prefrontal brain regions in components of working memory: effects of memory load and individual differences. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[53]  J. Fiez,et al.  Functional Magnetic Resonance Imaging (fmri) Was Used to Investigate the Neural Substrates of Component Processes in Verbal Working Memory. Based on Behavioral Research Using , 2022 .

[54]  Douglas N Greve,et al.  Identifying regional activity associated with temporally separated components of working memory using event-related functional MRI , 2003, NeuroImage.

[55]  J. Ashburner,et al.  Nonlinear spatial normalization using basis functions , 1999, Human brain mapping.

[56]  R. Krakow,et al.  Deaf signers and serial recall in the visual modality: Memory for signs, fingerspelling, and print , 1985, Memory & cognition.

[57]  G. Hickok,et al.  Neural substrates for verbal working memory in deaf signers: fMRI study and lesion case report , 2005, Brain and Language.

[58]  Carolyn C. Meltzer,et al.  Verbal and spatial working memory in older individuals: A positron emission tomography study , 2006, Brain Research.

[59]  T. Shallice,et al.  Please Scroll down for Article the Quarterly Journal of Experimental Psychology Independent Functioning of Verbal Memory Stores: a Neuropsychological Study Independent Functioning of Verbal Memory Stores : a Neuropsychological Study , 2022 .

[60]  From Neuropsychology to Mental Structure: The Short-Term Memory Syndrome , 1988 .

[61]  Mark D'Esposito,et al.  Dissociating Age-related Changes in Cognitive Strategy and Neural Efficiency Using Event- related fMRI , 2005, Cortex.

[62]  M. D’Esposito,et al.  The Influence of Working-Memory Demand and Subject Performance on Prefrontal Cortical Activity , 2002, Journal of Cognitive Neuroscience.

[63]  T Shallice,et al.  The selective impairment of auditory verbal short-term memory. , 1969, Brain : a journal of neurology.

[64]  R. Passingham,et al.  Active maintenance in prefrontal area 46 creates distractor-resistant memory , 2002, Nature Neuroscience.

[65]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .