Uncovering the architecture of action semantics.

Despite research suggesting that stored sensorimotor information about tool use is a component of the semantic representations of tools, little is known about the action features or organizing principles that underlie this knowledge. We used methods similar to those applied in other semantic domains to examine the "architecture" of action semantic knowledge. In Experiment 1, participants sorted photographs of tools into groups according to the similarity of their associated "use" actions and rated tools on dimensions related to action. The results suggest that the magnitude of arm movement, configuration of the hand, and manner of motion during tool use play a role in determining how tools cluster in action "semantic space." In Experiment 2, we validated the architecture uncovered in Experiment 1 using an implicit semantic task for which tool use knowledge was not ostensibly relevant (blocked cyclic word-picture matching). Using stimuli from Experiment 1, we found that participants performed more poorly during blocks of trials containing tools used with similar versus unrelated actions, and the amount of semantic interference depended on the magnitude of action similarity among tools. Thus, the degree of featural overlap between tool use actions plays a role in determining the overall semantic similarity of tools.

[1]  M. Brett,et al.  Actions Speak Louder Than Functions: The Importance of Manipulability and Action in Tool Representation , 2003, Journal of Cognitive Neuroscience.

[2]  M. Jeannerod Neural Simulation of Action: A Unifying Mechanism for Motor Cognition , 2001, NeuroImage.

[3]  S. Cappa,et al.  The breakdown of semantic knowledge: Insights from a statistical model of meaning representation , 2003, Brain and Language.

[4]  Alex Martin,et al.  Grounding Object Concepts in Perception and Action: Evidence from FMRI Studies of Tools , 2007, Cortex.

[5]  L. Buxbaum,et al.  Action knowledge, visuomotor activation, and embodiment in the two action systems , 2010, Annals of the New York Academy of Sciences.

[6]  L. Buxbaum,et al.  Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging. , 2005, Brain research. Cognitive brain research.

[7]  G. Lakoff,et al.  The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge , 2005, Cognitive neuropsychology.

[8]  Friedemann Pulverm Uuml,et al.  Words in the brain's language , 1999 .

[9]  George S. Cree,et al.  Evocation of functional and volumetric gestural knowledge by objects and words , 2008, Cognition.

[10]  W. Levelt,et al.  Semantic distance effects on object and action naming , 2002, Cognition.

[11]  Bradford Z. Mahon,et al.  A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content , 2008, Journal of Physiology-Paris.

[12]  W. K. Simmons,et al.  A common neural substrate for perceiving and knowing about color , 2007, Neuropsychologia.

[13]  Colin M. Macleod Half a century of research on the Stroop effect: an integrative review. , 1991, Psychological bulletin.

[14]  Susan J. Lederman,et al.  Cognitive representations of functional interactions with objects , 1993, Memory & cognition.

[15]  David C. Plaut,et al.  Semantic and Associative Priming in a Distributed Attractor Network , 1995 .

[16]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[17]  A. Glenberg,et al.  Processing Abstract Language Modulates Motor System Activity , 2008, Quarterly journal of experimental psychology.

[18]  Rolf A. Zwaan,et al.  Lateralization of object-shape information in semantic processing , 2004, Cognition.

[19]  Laurel J Buxbaum,et al.  Critical brain regions for action recognition: lesion symptom mapping in left hemisphere stroke. , 2010, Brain : a journal of neurology.

[20]  Marc W Howard,et al.  When Does Semantic Similarity Help Episodic Retrieval , 2002 .

[21]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[22]  D. V. von Cramon,et al.  Separable neuronal circuitries for manipulable and non-manipulable objects in working memory. , 2002, Cerebral cortex.

[23]  Ian S. Hargreaves,et al.  The neural correlates of the body-object interaction effect in semantic processing , 2012, Front. Hum. Neurosci..

[24]  Julie C. Sedivy,et al.  Impaired access to manipulation features in Apraxia: Evidence from eyetracking and semantic judgment tasks , 2010, Brain and Language.

[25]  M. Schwartz,et al.  Semantic interference during blocked-cyclic naming: Evidence from aphasia , 2006 .

[26]  W. Glaser,et al.  Context effects in stroop-like word and picture processing. , 1989, Journal of experimental psychology. General.

[27]  W. Levelt,et al.  Effects of semantic context in the naming of pictures and words , 2001, Cognition.

[28]  Eva Belke Effects of working memory load on lexical-semantic encoding in language production , 2008, Psychonomic bulletin & review.

[29]  B. P. McCloskey,et al.  Knowledge about hand shaping and knowledge about objects. , 1987, Journal of motor behavior.

[30]  Anna M. Borghi,et al.  Conceptual information about size of objects in nouns , 2009 .

[31]  Hisashi Q. Higuchi On the nature of , 1999 .

[32]  Kim Wilson,et al.  The Benefits of Sensorimotor Knowledge: Body-Object Interaction Facilitates Semantic Processing , 2008, Cogn. Sci..

[33]  M. Brodeur,et al.  The Bank of Standardized Stimuli (BOSS), a New Set of 480 Normative Photos of Objects to Be Used as Visual Stimuli in Cognitive Research , 2010, PloS one.

[34]  D. Bub,et al.  On the dynamics of action representations evoked by names of manipulable objects. , 2012, Journal of experimental psychology. General.

[35]  Eva Belke Long-lasting inhibitory semantic context effects on object naming are necessarily conceptually mediated: Implications for models of lexical-semantic encoding , 2013 .

[36]  J. Haxby,et al.  fMRI Responses to Video and Point-Light Displays of Moving Humans and Manipulable Objects , 2003, Journal of Cognitive Neuroscience.

[37]  Target-related distractors disrupt object selection in everyday action: Evidence from participants with dementia , 2010, Journal of the International Neuropsychological Society.

[38]  Michael Andres,et al.  Dissociation between manipulation and conceptual knowledge of object use in the supramarginalis gyrus , 2011, Human brain mapping.

[39]  G. Rizzolatti,et al.  The functional role of the parieto-frontal mirror circuit: interpretations and misinterpretations , 2010, Nature Reviews Neuroscience.

[40]  L. Buxbaum,et al.  Knowledge of object manipulation and object function: dissociations in apraxic and nonapraxic subjects , 2002, Brain and Language.

[41]  Luciano Fadiga,et al.  Hand action preparation influences the responses to hand pictures , 2002, Neuropsychologia.

[42]  Myrna F. Schwartz,et al.  Function and manipulation tool knowledge in apraxia: Knowing ‘what for’ but not ‘how’ , 2000 .

[43]  Julie C. Sedivy,et al.  Playing on the typewriter, typing on the piano: manipulation knowledge of objects , 2006, Cognition.

[44]  Tim Shallice,et al.  Refractoriness and the healthy brain: A behavioural study on semantic access , 2011, Cognition.

[45]  Markus Graf,et al.  The role of action representations in visual object recognition , 2006, Experimental Brain Research.

[46]  L. Buxbaum,et al.  Action matters: The role of action plans and object affordances in selection for action , 2002 .

[47]  Stephen McAdams,et al.  Comparison of Methods for Collecting and Modeling Dissimilarity Data: Applications to Complex Sound Stimuli , 2011, Multivariate behavioral research.

[48]  Eva Belke,et al.  Cumulative and Non-Cumulative Semantic Interference in Object Naming: Evidence from Blocked and Continuous Manipulations of Semantic Context , 2013, Quarterly journal of experimental psychology.

[49]  Chris McNorgan,et al.  An attractor model of lexical conceptual processing: simulating semantic priming , 1999, Cogn. Sci..

[50]  T. Shallice,et al.  Category specific semantic impairments , 1984 .

[51]  Mark S. Seidenberg,et al.  On the nature and scope of featural representations of word meaning. , 1997, Journal of experimental psychology. General.

[52]  D. Mirman,et al.  Abnormal dynamics of activation of object use information in apraxia: Evidence from eyetracking , 2014, Neuropsychologia.

[53]  David P Vinson,et al.  Semantic feature production norms for a large set of objects and events , 2008, Behavior research methods.

[54]  M. Kiefer,et al.  Action observation can prime visual object recognition , 2009, Experimental Brain Research.

[55]  Gary M. Oppenheim,et al.  The dark side of incremental learning: A model of cumulative semantic interference during lexical access in speech production , 2010, Cognition.

[56]  Bradford Z. Mahon,et al.  Lexical selection is not by competition: a reinterpretation of semantic interference and facilitation effects in the picture-word interference paradigm. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[57]  S. Thompson-Schill,et al.  Function follows form: activation of shape and function features during object identification. , 2011, Journal of experimental psychology. General.

[58]  Mark S. Seidenberg,et al.  Semantic feature production norms for a large set of living and nonliving things , 2005, Behavior research methods.

[59]  Olaf B. Paulson,et al.  When Action Turns into Words. Activation of Motor-Based Knowledge during Categorization of Manipulable Objects , 2002, Journal of Cognitive Neuroscience.

[60]  Harold Bekkering,et al.  Body schematics: On the role of the body schema in embodied lexical–semantic representations , 2010, Neuropsychologia.

[61]  M. Garrett,et al.  Representing the meanings of object and action words: The featural and unitary semantic space hypothesis , 2004, Cognitive Psychology.

[62]  Susan T. Dumais,et al.  A solution to Plato''s problem: representation of knowledge , 1997 .

[63]  A. Meyer,et al.  Refractory effects in picture naming as assessed in a semantic blocking paradigm , 2005, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[64]  Willem J. M. Levelt,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[65]  B. Mesquita,et al.  Adjustment to Chronic Diseases and Terminal Illness Health Psychology : Psychological Adjustment to Chronic Disease , 2006 .

[66]  M. Brysbaert,et al.  Age-of-acquisition ratings for 30,000 English words , 2012, Behavior research methods.

[67]  Robert Schreuder,et al.  Effects of perceptual and conceptual similarity in semantic priming , 1984 .

[68]  T. Shallice,et al.  Manipulability and object recognition: is manipulability a semantic feature? , 2011, Experimental Brain Research.

[69]  R. Ellis,et al.  On the relations between seen objects and components of potential actions. , 1998, Journal of experimental psychology. Human perception and performance.

[70]  M. Coltheart,et al.  Cumulative semantic inhibition in picture naming: experimental and computational studies , 2006, Cognition.

[71]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[72]  L. Buxbaum,et al.  Critical brain regions for tool-related and imitative actions: a componential analysis. , 2014, Brain : a journal of neurology.

[73]  D L Medin,et al.  Concepts and conceptual structure. , 1989, The American psychologist.

[74]  Michael Andres,et al.  Distinct contribution of the parietal and temporal cortex to hand configuration and contextual judgements about tools , 2013, Cortex.

[75]  J. Kroll,et al.  Category Interference in Translation and Picture Naming: Evidence for Asymmetric Connections Between Bilingual Memory Representations , 1994 .

[76]  Eiling Yee,et al.  Colorless Green Ideas (Can) Prime Furiously , 2012, Psychological science.

[77]  G. Rizzolatti,et al.  Action for perception: a motor-visual attentional effect. , 1999 .

[78]  Daniel N Bub,et al.  Features of Planned Hand Actions Influence Identification of Graspable Objects , 2013, Psychological science.

[79]  Susan T. Dumais,et al.  The latent semantic analysis theory of knowledge , 1997 .

[80]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[81]  Steven A. Jax,et al.  Response interference between functional and structural object-related actions is increased in patients with ideomotor apraxia. , 2013, Journal of neuropsychology.

[82]  A. A. Wijers,et al.  Visual semantic features are activated during the processing of concrete words: event-related potential evidence for perceptual semantic priming. , 2000, Brain research. Cognitive brain research.

[83]  M. L. Lambon Ralph,et al.  Conceptual Structure within and between Modalities , 2012, Front. Hum. Neurosci..

[84]  R. Marshall 5. Multidimensional Scaling. 2nd edn. Trevor F. Cox and Michael A. A. Cox, Chapman & Hall/CRC, Boca Raton, London, New York, Washington DC, 2000. No. of pages: xiv + 309. Price: $79.95. ISBN 1‐58488‐094‐5 , 2002 .

[85]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[86]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[87]  Solène Kalénine,et al.  Object manipulability affects children’s and adults’ conceptual processing , 2008, Psychonomic bulletin & review.

[88]  Murray Grossman,et al.  Left Inferior Parietal Representations for Skilled Hand-Object Interactions: Evidence from Stroke and Corticobasal Degeneration , 2007, Cortex.

[89]  Steven A. Jax,et al.  Response interference between functional and structural actions linked to the same familiar object , 2010, Cognition.

[90]  Lauren M. Bylsma,et al.  Toward an integrated account of object and action selection: A computational analysis and empirical findings from reaching-to-grasp and tool-use , 2009, Neuropsychologia.

[91]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[92]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[93]  Rolf A. Zwaan,et al.  Language Comprehenders Mentally Represent the Shapes of Objects , 2002, Psychological science.

[94]  D. Mirman,et al.  Incidental and context-responsive activation of structure- and function-based action features during object identification. , 2013, Journal of experimental psychology. Human perception and performance.

[95]  R. Klatzky,et al.  Cognitive representations of hand posture in ideomotor apraxia , 2003, Neuropsychologia.

[96]  Penny M. Pexman,et al.  Evidence for the activation of sensorimotor information during visual word recognition: The body–object interaction effect , 2008, Cognition.

[97]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[98]  W. Schneider,et al.  Perceptual Knowledge Retrieval Activates Sensory Brain Regions , 2006, The Journal of Neuroscience.

[99]  Antje S. Meyer,et al.  Exploring the time course of lexical access in language production : Picture word interference studies , 1990 .

[100]  D. Westwood,et al.  Manipulability and living/non-living category effects on object identification , 2005, Brain and Cognition.

[101]  Scott T. Grafton,et al.  Premotor Cortex Activation during Observation and Naming of Familiar Tools , 1997, NeuroImage.

[102]  Michael P. Kaschak,et al.  The Action-Sentence Compatibility Effect: It's All in the Timing , 2006, Cogn. Sci..

[103]  Randi C. Martin,et al.  Consequences of an inhibition deficit for word production and comprehension: Evidence from the semantic blocking paradigm , 2008, Cognitive neuropsychology.

[104]  Daniel N. Bub,et al.  Language-Based Access to Gestural Components of Conceptual Knowledge , 2008, Quarterly journal of experimental psychology.

[105]  Matthew H. Davis,et al.  Morphological and semantic effects in visual word recognition: A time-course study , 2000 .