Objects tell us what action we can expect: dissociating brain areas for retrieval and exploitation of action knowledge during action observation in fMRI

Objects are reminiscent of actions often performed with them: knife and apple remind us on peeling the apple or cutting it. Mnemonic representations of object-related actions (action codes) evoked by the sight of an object may constrain and hence facilitate recognition of unrolling actions. The present fMRI study investigated if and how action codes influence brain activation during action observation. The average number of action codes (NAC) of 51 sets of objects was rated by a group of n = 24 participants. In an fMRI study, different volunteers were asked to recognize actions performed with the same objects presented in short videos. To disentangle areas reflecting the storage of action codes from those exploiting them, we showed object-compatible and object-incompatible (pantomime) actions. Areas storing action codes were considered to positively co-vary with NAC in both object-compatible and object-incompatible action; due to its role in tool-related tasks, we here hypothesized left anterior inferior parietal cortex (aIPL). In contrast, areas exploiting action codes were expected to show this correlation only in object-compatible but not incompatible action, as only object-compatible actions match one of the active action codes. For this interaction, we hypothesized ventrolateral premotor cortex (PMv) to join aIPL due to its role in biasing competition in IPL. We found left anterior intraparietal sulcus (IPS) and left posterior middle temporal gyrus (pMTG) to co-vary with NAC. In addition to these areas, action codes increased activity in object-compatible action in bilateral PMv, right IPS, and lateral occipital cortex (LO). Findings suggest that during action observation, the brain derives possible actions from perceived objects, and uses this information to shape action recognition. In particular, the number of expectable actions quantifies the activity level at PMv, IPL, and pMTG, but only PMv reflects their biased competition while observed action unfolds.

[1]  R. Shaw,et al.  Perceiving, Acting and Knowing : Toward an Ecological Psychology , 1978 .

[2]  Speak Louder,et al.  Actions speak louder. , 1979, Hospital progress.

[3]  G. Fein Pretend play in childhood: An integrative review. , 1981 .

[4]  A. Tversky,et al.  Variants of uncertainty , 1982, Cognition.

[5]  K. Heilman,et al.  Ideational apraxia: A deficit in tool selection and use , 1989, Annals of neurology.

[6]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[7]  J. Decety,et al.  Brain structures participating in mental simulation of motor behavior: a neuropsychological interpretation. , 1990, Acta psychologica.

[8]  Ravi S. Menon,et al.  Imaging at high magnetic fields: initial experiences at 4 T. , 1993, Magnetic resonance quarterly.

[9]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[10]  H. Sakata,et al.  Deficit of hand preshaping after muscimol injection in monkey parietal cortex , 1994, Neuroreport.

[11]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited , 1995, NeuroImage.

[12]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited—Again , 1995, NeuroImage.

[13]  H. Sakata,et al.  Parietal neurons related to memory-guided hand manipulation. , 1996, Journal of neurophysiology.

[14]  Paul B. Johnson,et al.  Cortical networks for visual reaching: physiological and anatomical organization of frontal and parietal lobe arm regions. , 1996, Cerebral cortex.

[15]  G. Rizzolatti,et al.  Object representation in the ventral premotor cortex (area F5) of the monkey. , 1997, Journal of neurophysiology.

[16]  Michael A. Arbib,et al.  Modeling parietal-premotor interactions in primate control of grasping , 1998, Neural Networks.

[17]  R. Ellis,et al.  On the relations between seen objects and components of potential actions. , 1998, Journal of experimental psychology. Human perception and performance.

[18]  G. Rizzolatti,et al.  Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex (area F5). , 1998, Novartis Foundation symposium.

[19]  G. Glover Deconvolution of Impulse Response in Event-Related BOLD fMRI1 , 1999, NeuroImage.

[20]  M. Jeannerod TO ACT OR NOT TO ACT : PERSPECTIVES ON THE REPRESENTATION OF ACTIONS , 1999 .

[21]  G. Rizzolatti,et al.  Action for perception: a motor-visual attentional effect. , 1999, Journal of experimental psychology. Human perception and performance.

[22]  M. Jeannerod The 25th Bartlett Lecture , 1999 .

[23]  V. Gallese Action representaion and the inferior parietal lobule , 2000 .

[24]  G. Rizzolatti,et al.  The Organization of the Frontal Motor Cortex. , 2000, News in physiological sciences : an international journal of physiology produced jointly by the International Union of Physiological Sciences and the American Physiological Society.

[25]  M W Greenlee,et al.  Human cortical areas underlying the perception of optic flow: brain imaging studies. , 2000, International review of neurobiology.

[26]  Nancy Kanwisher,et al.  Implied motion activates extrastriate motion-processing areas Response to David and Senior (2000) , 2000, Trends in Cognitive Sciences.

[27]  H Burton,et al.  Attending to and Remembering Tactile Stimuli: A Review of Brain Imaging Data and Single-Neuron Responses , 2000, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[28]  Joanna McGrenere,et al.  Affordances: Clarifying and Evolving a Concep , 2000, Graphics Interface.

[29]  Joanna McGrenere,et al.  Affordances: Clarifying and Evolving a Concep , 2000, Graphics Interface.

[30]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[31]  D G Norris,et al.  Reduced power multislice MDEFT imaging , 2000, Journal of magnetic resonance imaging : JMRI.

[32]  J. Decety,et al.  Functional anatomy of execution, mental simulation, observation, and verb generation of actions: A meta‐analysis , 2001, Human brain mapping.

[33]  R. Ellis,et al.  The potentiation of grasp types during visual object categorization , 2001 .

[34]  G Lohmann,et al.  LIPSIA--a new software system for the evaluation of functional magnetic resonance images of the human brain. , 2001, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[35]  Leslie G. Ungerleider,et al.  The neural basis of biased competition in human visual cortex , 2001, Neuropsychologia.

[36]  C. Frith A framework for studying the neural basis of attention , 2001, Neuropsychologia.

[37]  R. Ward,et al.  S-R correspondence effects of irrelevant visual affordance: Time course and specificity of response activation , 2002 .

[38]  Yale E. Cohen,et al.  A common reference frame for movement plans in the posterior parietal cortex , 2002, Nature Reviews Neuroscience.

[39]  Emad N Eskandar,et al.  Distinct nature of directional signals among parietal cortical areas during visual guidance. , 2002, Journal of neurophysiology.

[40]  L. Buxbaum,et al.  Knowledge of object manipulation and object function: dissociations in apraxic and nonapraxic subjects , 2002, Brain and Language.

[41]  L. Buxbaum,et al.  Action matters: The role of action plans and object affordances in selection for action , 2002 .

[42]  K. Zilles,et al.  Crossmodal Processing of Object Features in Human Anterior Intraparietal Cortex An fMRI Study Implies Equivalencies between Humans and Monkeys , 2002, Neuron.

[43]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[44]  J. Danckert Common Mechanisms in Perception and Action: Attention and Performance XIX Wolfgang Prinz, Bernhard Hommel (Eds.), Oxford University Press, 2002, Price: £ 65.00, ISBN: 0-19-851069 , 2003, Neuropsychologia.

[45]  M. Bar A Cortical Mechanism for Triggering Top-Down Facilitation in Visual Object Recognition , 2003, Journal of Cognitive Neuroscience.

[46]  M. Brett,et al.  Actions Speak Louder Than Functions: The Importance of Manipulability and Action in Tool Representation , 2003, Journal of Cognitive Neuroscience.

[47]  R. Klatzky,et al.  Cognitive representations of hand posture in ideomotor apraxia , 2003, Neuropsychologia.

[48]  M. Wallace,et al.  Visual Localization Ability Influences Cross-Modal Bias , 2003, Journal of Cognitive Neuroscience.

[49]  Leslie G. Ungerleider,et al.  Neuroimaging Studies of Attention: From Modulation of Sensory Processing to Top-Down Control , 2003, The Journal of Neuroscience.

[50]  Paul Cisek,et al.  Mechanisms of selection and guidance of reaching movements in the parietal lobe. , 2003, Advances in neurology.

[51]  Steve Geraghty Seeing it my way , 2003 .

[52]  M. Rushworth,et al.  The left parietal and premotor cortices: motor attention and selection , 2003, NeuroImage.

[53]  R. Ellis,et al.  Action priming by briefly presented objects. , 2004, Acta psychologica.

[54]  D. Perrett,et al.  Opinion TRENDS in Cognitive Sciences Vol.8 No.11 November 2004 Demystifying social cognition: a Hebbian perspective , 2022 .

[55]  O. Blanke,et al.  Out-of-body experience and autoscopy of neurological origin. , 2004, Brain : a journal of neurology.

[56]  Gereon R. Fink,et al.  Neural basis of pantomiming the use of visually presented objects , 2004, NeuroImage.

[57]  J. Isnard,et al.  Surgical anatomy of the insula. , 2004, Advances and technical standards in neurosurgery.

[58]  G. Rizzolatti,et al.  Neurons related to goal-directed motor acts in inferior area 6 of the macaque monkey , 2004, Experimental Brain Research.

[59]  Scott H. Johnson-Frey The neural bases of complex tool use in humans , 2004, Trends in Cognitive Sciences.

[60]  Shy Shoham,et al.  Neural substrates of tactile object recognition: An fMRI study , 2004, Human brain mapping.

[61]  J. Kalaska,et al.  Neural Correlates of Reaching Decisions in Dorsal Premotor Cortex: Specification of Multiple Direction Choices and Final Selection of Action , 2005, Neuron.

[62]  G. Fink,et al.  REVIEW: The functional organization of the intraparietal sulcus in humans and monkeys , 2005, Journal of anatomy.

[63]  L. Buxbaum,et al.  Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging. , 2005, Brain research. Cognitive brain research.

[64]  Laurel J. Buxbaum,et al.  Deficient internal models for planning hand–object interactions in apraxia , 2005, Neuropsychologia.

[65]  D. Yves von Cramon,et al.  Variants of uncertainty in decision-making and their neural correlates , 2005, Brain Research Bulletin.

[66]  G. Orban,et al.  Specificity of regions processing biological motion , 2005, The European journal of neuroscience.

[67]  N. Kanwisher,et al.  Domain specificity in visual cortex. , 2006, Cerebral cortex.

[68]  Anjan Chatterjee,et al.  Specificity of Action Representations in the Lateral Occipitotemporal Cortex , 2006, Journal of Cognitive Neuroscience.

[69]  Julie C. Sedivy,et al.  Playing on the typewriter, typing on the piano: manipulation knowledge of objects , 2006, Cognition.

[70]  R. Ellis,et al.  The potentiation of two components of the reach-to-grasp action during object categorisation in visual memory. , 2006, Acta psychologica.

[71]  Alison J. Wiggett,et al.  The role of the extrastriate body area in action perception , 2006, Social neuroscience.

[72]  Markus Graf,et al.  The role of action representations in visual object recognition , 2006, Experimental Brain Research.

[73]  Kenneth F. Valyear,et al.  Human parietal cortex in action , 2006, Current Opinion in Neurobiology.

[74]  Alison J. Wiggett,et al.  Behavioral / Systems / Cognitive Functional Magnetic Resonance Imaging Investigation of Overlapping Lateral Occipitotemporal Activations Using Multi-Voxel Pattern Analysis , 2006 .

[75]  R. Schubotz Prediction of external events with our motor system: towards a new framework , 2007, Trends in Cognitive Sciences.

[76]  V. Ramachandran,et al.  Apraxia, metaphor and mirror neurons. , 2007, Medical hypotheses.

[77]  Jessie Chen,et al.  Neurophysiology of prehension. III. Representation of object features in posterior parietal cortex of the macaque monkey. , 2007, Journal of neurophysiology.

[78]  B. Mesquita,et al.  Adjustment to Chronic Diseases and Terminal Illness Health Psychology : Psychological Adjustment to Chronic Disease , 2006 .

[79]  Alison J. Wiggett,et al.  Functional MRI analysis of body and body part representations in the extrastriate and fusiform body areas. , 2007, Journal of neurophysiology.

[80]  Jessie Chen,et al.  Neurophysiology of prehension. I. Posterior parietal cortex and object-oriented hand behaviors. , 2007, Journal of neurophysiology.

[81]  U. Castiello,et al.  Differential cortical activity for precision and whole‐hand visually guided grasping in humans , 2007, The European journal of neuroscience.

[82]  Mitsuo Kawato,et al.  Central control of grasp: Manipulation of objects with complex and simple dynamics , 2007, NeuroImage.

[83]  Bradford Z. Mahon,et al.  Action-Related Properties Shape Object Representations in the Ventral Stream , 2007, Neuron.

[84]  J. Decety,et al.  The Role of the Right Temporoparietal Junction in Social Interaction: How Low-Level Computational Processes Contribute to Meta-Cognition , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[85]  Ed Symes,et al.  Visual object affordances: object orientation. , 2007, Acta psychologica.

[86]  Umberto Sabatini,et al.  Neural bases of personal and extrapersonal neglect in humans. , 2007, Brain : a journal of neurology.

[87]  G. Sheean,et al.  Upper Motor Neurone Syndrome and Spasticity: Neurophysiology of spasticity , 2008 .

[88]  三嶋 博之 The theory of affordances , 2008 .

[89]  Paul T. Sowden,et al.  Your hand or mine? The extrastriate body area , 2008, NeuroImage.

[90]  Jason P. Mitchell Activity in right temporo-parietal junction is not selective for theory-of-mind. , 2008, Cerebral cortex.

[91]  R. Andersen,et al.  Intention, Action Planning, and Decision Making in Parietal-Frontal Circuits , 2009, Neuron.

[92]  G. Kreiman,et al.  Timing, Timing, Timing: Fast Decoding of Object Information from Intracranial Field Potentials in Human Visual Cortex , 2009, Neuron.

[93]  W. Heide,et al.  Visual Search Disorders in Acute and Chronic Homonymous Hemianopia , 2009, Annals of the New York Academy of Sciences.

[94]  Frank Van Overwalle,et al.  Understanding others' actions and goals by mirror and mentalizing systems: A meta-analysis , 2009, NeuroImage.

[95]  D. Yves von Cramon,et al.  The Case of Pretense: Observing Actions and Inferring Goals , 2009, Journal of Cognitive Neuroscience.

[96]  Sarah H. Creem-Regehr,et al.  Sensory-motor and cognitive functions of the human posterior parietal cortex involved in manual actions , 2009, Neurobiology of Learning and Memory.

[97]  Hansjörg Scherberger,et al.  Context-Specific Grasp Movement Representation in the Macaque Anterior Intraparietal Area , 2009, The Journal of Neuroscience.

[98]  F. Van Overwalle Social cognition and the brain: A meta‐analysis , 2009, Human brain mapping.

[99]  R. Nicoletti,et al.  Simon-Like and Functional Affordance Effects with Tools: The Effects of Object Perceptual Discrimination and Object Action State , 2010, Quarterly journal of experimental psychology.

[100]  Angela R. Laird,et al.  ALE meta-analysis of action observation and imitation in the human brain , 2010, NeuroImage.

[101]  James B. Rowe,et al.  Action selection: A race model for selected and non-selected actions distinguishes the contribution of premotor and prefrontal areas , 2010, NeuroImage.

[102]  R. Proctor,et al.  The object-based Simon effect: grasping affordance or relative location of the graspable part? , 2010, Journal of experimental psychology. Human perception and performance.

[103]  R. Nicoletti,et al.  On the relationship between affordance and Simon effects: Are the effects really independent? , 2011 .

[104]  T. Shallice,et al.  Manipulability and object recognition: is manipulability a semantic feature? , 2011, Experimental Brain Research.

[105]  M. L. Ralph,et al.  Different roles of lateral anterior temporal lobe and inferior parietal lobule in coding function and manipulation tool knowledge: Evidence from an rTMS study , 2011, Neuropsychologia.

[106]  E. Wagenmakers,et al.  Erroneous analyses of interactions in neuroscience: a problem of significance , 2011, Nature Neuroscience.

[107]  Alice Mado Proverbio,et al.  250ms to code for action affordance during observation of manipulable objects , 2011, Neuropsychologia.

[108]  Petroc Sumner,et al.  Conflict in object affordance revealed by grip force , 2012, Quarterly journal of experimental psychology.

[109]  A. Caramazza,et al.  Brain Regions That Represent Amodal Conceptual Knowledge , 2013, The Journal of Neuroscience.

[110]  F. Binkofski,et al.  Two action systems in the human brain , 2013, Brain and Language.

[111]  Peter C. Hansen,et al.  Seeing It My Way or Your Way: Frontoparietal Brain Areas Sustain Viewpoint-independent Perspective Selection Processes , 2013, Journal of Cognitive Neuroscience.

[112]  Sabine Kastner,et al.  The representation of tool and non-tool object information in the human intraparietal sulcus. , 2013, Journal of neurophysiology.

[113]  Alison J. Clarke,et al.  “Actions Speak Louder” , 2013 .

[114]  Bill Wilson,et al.  In a relationship , 2013 .

[115]  G. Fink,et al.  The role of the right temporoparietal junction in attention and social interaction as revealed by ALE meta-analysis , 2014, Brain Structure and Function.

[116]  Kenneth M. Heilman,et al.  Apraxia : The Neuropsychology of Action , 2014 .