Sensory load incurs conceptual processing costs

Theories of grounded cognition propose that modal simulations underlie cognitive representation of concepts [Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577-660; Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617-645]. Based on recent evidence of modality-specific resources in perception, we hypothesized that verifying properties of concepts encoded in different modalities are hindered more by perceptual short-term memory load to the same versus different sensory modality as that used to process the property. We manipulated load to visual and auditory modalities by having participants store one or three items in short-term memory during property verification. In the high (but not low) load condition, property verification took longer when the property (e.g., yellow) involved the same modality as that used by the memory load (e.g., pictures). Interestingly, similar interference effects were obtained on the conceptual verification and on the memory task. These findings provide direct support for the view that conceptual processing relies on simulation in modality-specific systems.

[1]  David Alais,et al.  Separate attentional resources for vision and audition , 2006, Proceedings of the Royal Society B: Biological Sciences.

[2]  J. Sweller,et al.  Reducing cognitive load by mixing auditory and visual presentation modes , 1995 .

[3]  W. K. Simmons,et al.  Pictures of appetizing foods activate gustatory cortices for taste and reward. , 2005, Cerebral cortex.

[4]  L. Barsalou,et al.  Verifying Different-Modality Properties for Concepts Produces Switching Costs , 2003, Psychological science.

[5]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[6]  J. K. Eddy,et al.  The verification of high and low imagery sentences , 1980 .

[7]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[8]  L. M. Warner,et al.  The Neural Mechanisms for Minimizing Cross-Modal Distraction , 2004, The Journal of Neuroscience.

[9]  P. Niedenthal Embodying Emotion , 2007, Science.

[10]  C. Spence,et al.  The cost of expecting events in the wrong sensory modality , 2001, Perception & psychophysics.

[11]  M. Woldorff,et al.  Attentional capacity for processing concurrent stimuli is larger across sensory modalities than within a modality. , 2006, Psychophysiology.

[12]  Jason B. Mattingley,et al.  Modality-Specific Control of Strategic Spatial Attention in Parietal Cortex , 2004, Neuron.

[13]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[14]  Nicolas Vermeulen,et al.  Switching Between Sensory and Affective Systems Incurs Processing Costs , 2007, Cogn. Sci..

[15]  Raymond Bruyer,et al.  Does visual short‐term memory store bound features? , 2004 .

[16]  Irene P. Kan,et al.  ROLE OF MENTAL IMAGERY IN A PROPERTY VERIFICATION TASK: FMRI EVIDENCE FOR PERCEPTUAL REPRESENTATIONS OF CONCEPTUAL KNOWLEDGE , 2003, Cognitive neuropsychology.

[17]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[18]  V. Gallese The Roots of Empathy: The Shared Manifold Hypothesis and the Neural Basis of Intersubjectivity , 2003, Psychopathology.

[19]  C. Frith,et al.  Processing of irrelevant visual motion during performance of an auditory attention task , 2001, Neuropsychologia.

[20]  Diane Pecher,et al.  Sensorimotor simulations underlie conceptual representations: Modality-specific effects of prior activation , 2004, Psychonomic bulletin & review.