Cross‐modal temporal order memory for auditory digits and visual locations: An fMRI study

A function of working memory is to remember the temporal sequence of events, often occurring across different sensory modalities. To study the neural correlates of this function, we conducted an event‐related functional magnetic resonance imaging (fMRI) experiment with a cross‐modal memory task. Subjects were required to recall auditory digits and visual locations either in mixed order (cross‐modality) or in separate order (within‐modality). To identify the brain regions involved in the memory of cross‐modal temporal order, we compared the blood oxygenation level‐dependent (BOLD) response between the mixed and the separate order tasks. As a control, cortical areas sensitive to the memory load were mapped by comparing the 10‐item condition with the 6‐item condition in the separate order task. Results show that the bilateral prefrontal, right premotor, temporo‐parietal junction (TPJ) and left superior parietal cortices had significantly more activation in the mixed task than in the separate task. Some of these areas were also sensitive to the memory load, whereas the right prefrontal cortex and TPJ were relatively more sensitive to the cross‐modal order but not the memory load. Our study provides potential neural correlates for the episodic buffer, a key component of working memory as proposed previously [Baddeley. Trends Cogn Sci 2000;4:417–423]. Hum. Brain Mapping 22:280–289, 2004. © 2004 Wiley‐Liss, Inc.

[1]  J. Fisher,et al.  Neuropsychological Assessment, 2nd Ed , 1985 .

[2]  C. Penney Modality effects and the structure of short-term verbal memory , 1989, Memory & cognition.

[3]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[4]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[5]  J. Mazziotta,et al.  Brain-behavior relationships: evidence from practice effects in spatial stimulus-response compatibility. , 1996, Journal of neurophysiology.

[6]  Hefei MIXED-MODALITY SPAN OF VISUOSPATIAL STIMULI AND AUDITORY DIGITS , 1997 .

[7]  A. Dale,et al.  Selective averaging of rapidly presented individual trials using fMRI , 1997, Human brain mapping.

[8]  A M Dale,et al.  Event-related functional MRI: past, present, and future. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[9]  J. Desmond,et al.  Dissociation of Frontal and Cerebellar Activity in a Cognitive Task: Evidence for a Distinction between Selection and Search , 1998, NeuroImage.

[10]  P. Roland,et al.  Right prefrontal activation during encoding, but not during retrieval, in a non-verbal paired-associates task. , 1998, Cerebral cortex.

[11]  B. Postle,et al.  Functional neuroanatomical double dissociation of mnemonic and executive control processes contributing to working memory performance. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[12]  M D'Esposito,et al.  The roles of prefrontal brain regions in components of working memory: effects of memory load and individual differences. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[13]  G. Glover Deconvolution of Impulse Response in Event-Related BOLD fMRI1 , 1999, NeuroImage.

[14]  Hefei,et al.  POSSIBLE ACCOUNT FOR THE DECREMENT OF RECALL ACCORDING TO TEMPORAL PRESENTATION ORDER INVISUOSPATIAL AND AUDITORY DUAL MEMORY TASK , 1999 .

[15]  R. Turner,et al.  Form and motion coherence activate independent, but not dorsal/ventral segregated, networks in the human brain , 2000, Current Biology.

[16]  S. Magnussen Low-level memory processes in vision , 2000, Trends in Neurosciences.

[17]  A. Baddeley The episodic buffer: a new component of working memory? , 2000, Trends in Cognitive Sciences.

[18]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[19]  M Naveh-Benjamin,et al.  The Effects of Divided Attention on Encoding and Retrieval Processes: The Resiliency of Retrieval Processes , 2000, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[20]  J. D. E. Gabrieli,et al.  Integration of diverse information in working memory within the frontal lobe , 2000, Nature Neuroscience.

[21]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[22]  J D Gabrieli,et al.  A resource model of the neural basis of executive working memory. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[23]  R. Passingham,et al.  The prefrontal cortex: response selection or maintenance within working memory? , 2000, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[24]  R. Knight,et al.  Prefrontal–cingulate interactions in action monitoring , 2000, Nature Neuroscience.

[25]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[26]  John Jonides,et al.  Order Information in Working Memory: fMRI Evidence for Parietal and Prefrontal Mechanisms , 2000, Journal of Cognitive Neuroscience.

[27]  S. Iversen,et al.  Detection of Audio-Visual Integration Sites in Humans by Application of Electrophysiological Criteria to the BOLD Effect , 2001, NeuroImage.

[28]  Daniel L. Schacter,et al.  Priming within and across Modalities: Exploring the Nature of rCBF Increases and Decreases , 2001, NeuroImage.

[29]  R. Buxton,et al.  Detection Power, Estimation Efficiency, and Predictability in Event-Related fMRI , 2001, NeuroImage.

[30]  James B. Rowe,et al.  Working Memory for Location and Time: Activity in Prefrontal Area 46 Relates to Selection Rather than Maintenance in Memory , 2001, NeuroImage.

[31]  E. Bullmore,et al.  Mapping Motor Inhibition: Conjunctive Brain Activations across Different Versions of Go/No-Go and Stop Tasks , 2001, NeuroImage.

[32]  J. Fiez,et al.  Functional Magnetic Resonance Imaging (fmri) Was Used to Investigate the Neural Substrates of Component Processes in Verbal Working Memory. Based on Behavioral Research Using , 2022 .

[33]  M. Hallett,et al.  Neural Correlates of Auditory–Visual Stimulus Onset Asynchrony Detection , 2001, The Journal of Neuroscience.

[34]  Joy Hirsch,et al.  Interconnected Large-Scale Systems for Three Fundamental Cognitive Tasks Revealed by Functional MRI , 2001, Journal of Cognitive Neuroscience.

[35]  G. Glover,et al.  Error‐related brain activation during a Go/NoGo response inhibition task , 2001, Human brain mapping.

[36]  Stephen M. Rao,et al.  The evolution of brain activation during temporal processing , 2001, Nature Neuroscience.

[37]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[38]  M. Corbetta,et al.  Neural Systems for Visual Orienting and Their Relationships to Spatial Working Memory , 2002, Journal of Cognitive Neuroscience.

[39]  H. Garavan,et al.  Dissociable Executive Functions in the Dynamic Control of Behavior: Inhibition, Error Detection, and Correction , 2002, NeuroImage.

[40]  Tyrone D. Cannon,et al.  Maintenance and Manipulation in Spatial Working Memory: Dissociations in the Prefrontal Cortex , 2002, NeuroImage.

[41]  M. Hallett,et al.  Neural correlates of cross-modal binding , 2003, Nature Neuroscience.

[42]  Joseph A Maldjian,et al.  Cross‐modal sensory processing in the anterior cingulate and medial prefrontal cortices , 2003, Human brain mapping.

[43]  Raymond J Dolan,et al.  Maintenance versus manipulation in verbal working memory revisited: an fMRI study , 2003, NeuroImage.

[44]  H Garavan,et al.  Co-ordination within and between verbal and visuospatial working memory: network modulation and anterior frontal recruitment , 2003, NeuroImage.

[45]  A. Baddeley Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.