Embodied cognition

In this presentation, we discuss embodied cognition in the human brain from perspectives of spatial cognition, sensorimotor processing, face processing, and mobile EEG recordings. The argument is based upon experimental evidence gathered from five separate studies. First, we focus on spatial representations and demonstrate that, given time pressure, information on the spatial orientation of houses, independent of a participant's own location, is best retrieved when it directly relates to potential actions. Thus providing evidence that even spatial representations code information in a manner directly related to the action. Next, we discuss the concept of representations as such. Using the example of face processing in the human visual system, we argue that the concept of representations should be confined to cases where neuronal activity contains explicit information on the variable of interest and, in turn, that this variable explains the complete part of the explainable variance, i.e. reaches the noise limit. Next, to push towards an investigation of cognition under natural conditions we present a benchmark test of mobile and research-grade EEG systems. Specifically, we demonstrate that the variance over systems contributes a significant part to the total variance of recorded event related potentials. As a next step, using Independent Component Analysis of EEG data we demonstrate that in cognitive tasks some independent components systematically relate to sensory processing as well as to action execution. This supports theories of the common coding theory and, thus, a mechanistic part of the embodied cognition framework. Finally, we demonstrate a real world application investigating face processing in the form of the N170 event related potential during natural visual exploration in a fully mobile setup. This technique allows investigating the physiological basis of cognitive processes under real world conditions. In this presentation we argue that understanding cognitive processes will need to consider the (inter)actions in the natural environment.

[1]  Esther Thelen,et al.  Dynamic Systems Theories , 2007 .

[2]  Ruth Garrett Millikan,et al.  Pushmi-Pullyu Representations , 1995 .

[3]  P. König,et al.  Where's the action? The pragmatic turn in cognitive science , 2013, Trends in Cognitive Sciences.

[4]  G. Rizzolatti,et al.  View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex , 2011, Current Biology.

[5]  Peter König,et al.  Are allocentric spatial reference frames compatible with theories of Enactivism? , 2017, Psychological Research.

[6]  Alexander Maye,et al.  Using Sensorimotor Contingencies for Prediction and Action Planning , 2012, SAB.

[7]  P. König,et al.  Beyond sensory substitution—learning the sixth sense , 2005, Journal of neural engineering.

[8]  G. Rizzolatti,et al.  Planning actions in autism , 2008, Experimental Brain Research.

[9]  Robert Liebendorfer Mind, self and society , 1960 .

[10]  Daniel P Ferris,et al.  EEG correlates of sensorimotor processing: independent components involved in sensory and motor processing , 2017, Scientific Reports.

[11]  Peter König,et al.  Sensory Augmentation for the Blind , 2012, Front. Hum. Neurosci..

[12]  Michael L. Anderson Embodied Cognition: A field guide , 2003, Artif. Intell..

[13]  Arne D. Ekstrom,et al.  Single-Neuron Responses in Humans during Execution and Observation of Actions , 2010, Current Biology.

[14]  A. Clark,et al.  The Extended Mind , 1998, Analysis.

[15]  D. Smith Consciousness in action , 2004, Synthese.

[16]  Alfonso Caramazza,et al.  Asymmetric fMRI adaptation reveals no evidence for mirror neurons in humans , 2009, Proceedings of the National Academy of Sciences.

[17]  Gaetano Tieri,et al.  Where does an object trigger an action? An investigation about affordances in space , 2010, Experimental Brain Research.

[18]  Tom Ziemke,et al.  Sentence Processing: Linking Language to Motor Chains , 2010, Front. Neurorobot..

[19]  S. Tipper,et al.  Vision-for-action: The effects of object property discrimination and action state on affordance compatibility effects , 2006, Psychonomic bulletin & review.

[20]  C. Pennartz Consciousness, Representation, Action: The Importance of Being Goal-Directed , 2018, Trends in Cognitive Sciences.

[21]  Glyn Lewis,et al.  Overcoming the emotion experience / expression dichotomy , 2012 .

[22]  Weimin Mou,et al.  Allocentric and egocentric updating of spatial memories. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[23]  Karl J. Friston,et al.  Evidence of Mirror Neurons in Human Inferior Frontal Gyrus , 2009, The Journal of Neuroscience.

[24]  W. David Hairston,et al.  Systems, Subjects, Sessions: To What Extent Do These Factors Influence EEG Data? , 2017, Front. Hum. Neurosci..

[25]  D Yves von Cramon,et al.  Sequences of Abstract Nonbiological Stimuli Share Ventral Premotor Cortex with Action Observation and Imagery , 2004, The Journal of Neuroscience.

[26]  T. Riley,et al.  Brain-Wise: Studies in Neurophilosophy , 2003 .

[27]  A. Craig,et al.  How do you feel — now? The anterior insula and human awareness , 2009, Nature Reviews Neuroscience.

[28]  S. Vereza Philosophy in the flesh: the embodied mind and its challenge to Western thought , 2001 .

[29]  R. Nicoletti,et al.  Simon-Like and Functional Affordance Effects with Tools: The Effects of Object Perceptual Discrimination and Object Action State , 2010, Quarterly journal of experimental psychology.

[30]  G. Carpenter,et al.  Behavioral and Brain Sciences , 1999 .

[31]  A. Glenberg,et al.  Symbol Grounding and Meaning: A Comparison of High-Dimensional and Embodied Theories of Meaning , 2000 .

[32]  Umberto Castiello,et al.  Social grasping: From mirroring to mentalizing , 2012, NeuroImage.

[33]  F. Binkofski,et al.  Two action systems in the human brain , 2013, Brain and Language.

[34]  V. Gallese,et al.  Bodily self and schizophrenia: The loss of implicit self-body knowledge , 2012, Consciousness and Cognition.

[35]  A. Goldman,et al.  Is social cognition embodied? , 2009, Trends in Cognitive Sciences.

[36]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[37]  Steve Torrance,et al.  In search of the enactive: Introduction to special issue on enactive experience , 2005 .

[38]  Bradford Z. Mahon,et al.  What drives the organization of object knowledge in the brain? , 2011, Trends in Cognitive Sciences.

[39]  V. Gallese Intentional attunement: A neurophysiological perspective on social cognition and its disruption in autism , 2006, Brain Research.

[40]  U. Castiello,et al.  Modulation of the action control system by social intention: unexpected social requests override preplanned action. , 2009, Journal of experimental psychology. Human perception and performance.

[41]  Sabine U. König,et al.  The experience of new sensorimotor contingencies by sensory augmentation , 2014, Consciousness and Cognition.

[42]  G. Rizzolatti,et al.  Impairment of actions chains in autism and its possible role in intention understanding , 2007, Proceedings of the National Academy of Sciences.

[43]  Shaun Gallagher,et al.  Body Schema and Intentionality , 1995 .

[44]  G. Ryle,et al.  The concept of mind. , 2004, The International journal of psycho-analysis.

[45]  G. Rizzolatti,et al.  Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex (area F5). , 1998, Novartis Foundation symposium.

[46]  Anna M. Borghi,et al.  Action language comprehension , affordances and goals . In , 2011 .

[47]  V. Gallese,et al.  The mirror mechanism and its potential role in autism spectrum disorder , 2013, Developmental medicine and child neurology.

[48]  Andy Clark,et al.  Vision as Dance? Three Challenges for Sensorimotor Contingency Theory , 2006 .

[49]  Lucia Riggio,et al.  Sentence comprehension and simulation of object temporary, canonical and stable affordances , 2009, Brain Research.

[50]  A. Craig How do you feel? Interoception: the sense of the physiological condition of the body , 2002, Nature Reviews Neuroscience.

[51]  Yann Coello,et al.  Language and action in cognitive neuroscience , 2012 .

[52]  H. Ruley,et al.  Observational fear learning involves affective pain system and Cav1.2 Ca2+ channels in ACC , 2010, Nature Neuroscience.

[53]  Leonardo Fogassi,et al.  Mirror neurons encode the subjective value of an observed action , 2012, Proceedings of the National Academy of Sciences.

[54]  Bradford Z. Mahon,et al.  A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content , 2008, Journal of Physiology-Paris.

[55]  Maurizio Gentilucci,et al.  The bottle and the glass say to me: “Pour!” , 2012, Experimental Brain Research.

[56]  Vittorio Gallese,et al.  Mirror neurons: A sensorimotor representation system , 2001 .

[57]  G. Rizzolatti,et al.  Mirror Neurons Differentially Encode the Peripersonal and Extrapersonal Space of Monkeys , 2009, Science.

[58]  Angelo Cangelosi,et al.  Object affordance influences instruction span , 2012, Experimental Brain Research.

[59]  K. Grill-Spector,et al.  Electrical Stimulation of Human Fusiform Face-Selective Regions Distorts Face Perception , 2012, The Journal of Neuroscience.

[60]  Kai-Uwe Kühnberger,et al.  A Unifying Approach to High- and Low-Level Cognition , 2013 .

[61]  Michael A. Arbib,et al.  Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons , 2010, Biological Cybernetics.

[62]  Vittorio Gallese,et al.  Listening to Action-related Sentences Activates Fronto-parietal Motor Circuits , 2005, Journal of Cognitive Neuroscience.

[63]  K. Jellinger Motor Cognition What Actions Tell the Self , 2007 .

[64]  A. Cangelosi,et al.  How affordances associated with a distractor object affect compatibility effects: A study with the computational model TRoPICALS , 2013, Psychological research.

[65]  Marina Basu The Embodied Mind: Cognitive Science and Human Experience , 2004 .

[66]  D. McDermott LANGUAGE OF THOUGHT , 2012 .

[67]  L. Wheaton,et al.  One hand, two objects: Emergence of affordance in contexts , 2012, Brain and Cognition.

[68]  A. Borghi Object concepts and action: extracting affordances from objects parts. , 2004, Acta psychologica.

[69]  G. Rizzolatti,et al.  Functional organization of inferior area 6 in the macaque monkey , 2004, Experimental Brain Research.

[70]  Michael P. Kaschak,et al.  Body posture facilitates retrieval of autobiographical memories , 2007, Cognition.

[71]  Edmund Husserl,et al.  Phänomenologische Untersuchungen zur Konstitution , 1952 .

[72]  N. Burgess,et al.  Spatial memory: how egocentric and allocentric combine , 2006, Trends in Cognitive Sciences.

[73]  Vittorio Gallese,et al.  Functional organization of the insula and inner perisylvian regions , 2012, Proceedings of the National Academy of Sciences.

[74]  H. Bekkering,et al.  Embodied Language Comprehension Requires an Enactivist Paradigm of Cognition , 2010, Front. Psychology.

[75]  Phil Husbands,et al.  Evolutionary robotics , 2014, Evolutionary Intelligence.

[76]  Martin H. Fischer,et al.  A hierarchical view of grounded, embodied, and situated numerical cognition , 2012, Cognitive Processing.

[77]  R. Ellis,et al.  The potentiation of grasp types during visual object categorization , 2001 .

[78]  Amir Amedi,et al.  Reading with Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in the Blind , 2012, Neuron.

[79]  Roberta L. Klatzky,et al.  Allocentric and Egocentric Spatial Representations: Definitions, Distinctions, and Interconnections , 1998, Spatial Cognition.

[80]  Rick Grush,et al.  The emulation theory of representation: Motor control, imagery, and perception , 2004, Behavioral and Brain Sciences.

[81]  Joseph Jastrow,et al.  THE WILL TO BELIEVE , 1922 .

[82]  A. Borghi,et al.  Object Affordances Tune Observers' Prior Expectations about Tool-Use Behaviors , 2012, PloS one.

[83]  F. Binkofski,et al.  Abstract and concrete phrases processing differentially modulates cortico-spinal excitability , 2012, Brain Research.

[84]  John Bridgeman,et al.  Bodies and other visual objects: the dialectics of reaching toward objects , 2013, Psychological research.

[85]  Andreas Lüthi,et al.  Mouse brains wired for empathy? , 2010, Nature Neuroscience.

[86]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[87]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[88]  Amy Coplan Simulating Minds: The Philosophy, Psychology, and Neuroscience of Mindreading by goldman, alvin , 2008 .

[89]  M. Graziano The organization of behavioral repertoire in motor cortex. , 2006, Annual review of neuroscience.

[90]  V. Gallese Mirror neurons and the social nature of language: The neural exploitation hypothesis , 2008, Social neuroscience.

[91]  Sylvia Weir,et al.  Action perception , 1974 .

[92]  Antonio R. Damasio Emozione e coscienza , 2000 .

[93]  P. Niedenthal Embodying Emotion , 2007, Science.

[94]  Jean-Luc Velay,et al.  Digitizing Literacy: Reflections on the Haptics of Writing , 2010 .

[95]  A. Pérez-Villalba Rhythms of the Brain, G. Buzsáki. Oxford University Press, Madison Avenue, New York (2006), Price: GB £42.00, p. 448, ISBN: 0-19-530106-4 , 2008 .

[96]  M. Costantini,et al.  When objects are close to me: Affordances in the peripersonal space , 2011, Psychonomic bulletin & review.

[97]  James L. McClelland,et al.  Phenomenology of perception. , 1978, Science.

[98]  S. Vocks,et al.  Body image disturbance in binge eating disorder: a review. , 2011, European eating disorders review : the journal of the Eating Disorders Association.

[99]  Giacomo Rizzolatti,et al.  So quel che fai : il cervello che agisce e i neuroni specchio , 2006 .

[100]  D. M. Hutton,et al.  Cambrian Intelligence: The Early History of the New AI , 2000 .

[101]  Paul Dourish,et al.  Where the action is , 2001 .

[102]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[103]  G. Pezzulo,et al.  When affordances climb into your mind: Advantages of motor simulation in a memory task performed by novice and expert rock climbers , 2010, Brain and Cognition.

[104]  Roel M. Willems,et al.  Embodied Cognition: Taking the Next Step , 2012, Front. Psychology.

[105]  Bradford Z. Mahon,et al.  The organization of conceptual knowledge: the evidence from category-specific semantic deficits , 2003, Trends in Cognitive Sciences.

[106]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[107]  G. Vallar,et al.  Somatoparaphrenia: a body delusion. A review of the neuropsychological literature , 2008, Experimental Brain Research.

[108]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[109]  Chris Letheby In defence of embodied cognition: a reply to Fred Adams , 2012 .

[110]  Christian Büchel,et al.  Learning New Sensorimotor Contingencies: Effects of Long-Term Use of Sensory Augmentation on the Brain and Conscious Perception , 2016, PloS one.

[111]  F. Strack,et al.  Inhibiting and facilitating conditions of the human smile: a nonobtrusive test of the facial feedback hypothesis. , 1988, Journal of personality and social psychology.

[112]  Fausto Caruana In principio era l'azione. Un'analisi normativa dei concetti nell'embodied simulation , 2006 .

[113]  S. Peters,et al.  Neural Correlates of Categorical Perception in Learned Vocal Communication , 2009, Nature Neuroscience.

[114]  Vittorio Gallese,et al.  Motor abstraction: a neuroscientific account of how action goals and intentions are mapped and understood , 2009, Psychological research.

[115]  Vittorio Gallese,et al.  How the motor system handles nouns: a behavioral study , 2011, Psychological Research.

[116]  V. Gallese,et al.  Intentional Attunement: Mirror Neurons and the Neural Underpinnings of Interpersonal Relations , 2007, Journal of the American Psychoanalytic Association.

[117]  D. Marocco,et al.  Manipulating Objects and Telling Words: A Study on Concrete and Abstract Words Acquisition , 2011, Front. Psychology.

[118]  D. Parisi,et al.  TRoPICALS : A Computational Embodied Neuroscience Model of Experiments on Compatibility Effects , 2010 .

[119]  Guy Dove On the need for Embodied and Dis-Embodied Cognition , 2011, Front. Psychology.

[120]  V. Gallese,et al.  Motor Representation of Actions in Children with Autism , 2012, PloS one.

[121]  Ivilin Peev Stoianov,et al.  When Action Meets Emotions: How Facial Displays of Emotion Influence Goal-Related Behavior , 2010, PloS one.

[122]  A. Glenberg,et al.  Action-based language: A theory of language acquisition, comprehension, and production , 2012, Cortex.

[123]  L. Fadiga,et al.  Active perception: sensorimotor circuits as a cortical basis for language , 2010, Nature Reviews Neuroscience.

[124]  L. Steels Modeling the cultural evolution of language. , 2011, Physics of life reviews.

[125]  Frank Tong,et al.  Representational Dynamics of Facial Viewpoint Encoding , 2017, Journal of Cognitive Neuroscience.

[126]  Christine D. Wilson,et al.  Grounding conceptual knowledge in modality-specific systems , 2003, Trends in Cognitive Sciences.

[127]  A Pupi,et al.  Looking at my body. Similarities and differences between anorexia nervosa patients and controls in body image visual processing , 2013, European Psychiatry.

[128]  A. Caramazza,et al.  The organisation of conceptual knowledge in the brain: The future's past and some future directions , 2006, Cognitive neuropsychology.

[129]  Klaus Gramann,et al.  Spatial Cognition & Computation: an Interdisciplinary Journal Embodiment of Spatial Reference Frames and Individual Differences in Reference Frame Proclivity Embodiment of Spatial Reference Frames and Individual Differences in Reference Frame Proclivity , 2022 .

[130]  Annabelle Blangero,et al.  Got it! Understanding the concept of a tool , 2010, NeuroImage.

[131]  Robert A. Wilson,et al.  Embodied cognition. , 2013, Wiley interdisciplinary reviews. Cognitive science.

[132]  Anna M. Borghi Parole, non simboli , 2010 .

[133]  Cosimo Urgesi,et al.  Long- and short-term plastic modeling of action prediction abilities in volleyball , 2011, Psychological Research.

[134]  Maria Alessandra Umiltà,et al.  Responses of mirror neurons in area F5 to hand and tool grasping observation , 2010, Experimental Brain Research.

[135]  Daniel D. Hutto Knowing What? Radical Versus Conservative Enactivism , 2005 .

[136]  G. Rizzolatti,et al.  Intention Understanding in Autism , 2009, PloS one.

[137]  G Vallar Tavola rotonda. Neuro-mania. Il cervello non spiega chi siamo , 2009 .

[138]  R. Ellis,et al.  Micro-affordance: the potentiation of components of action by seen objects. , 2000, British journal of psychology.

[139]  Charles Wallis,et al.  Computation and cognition , 2003, J. Exp. Theor. Artif. Intell..

[140]  G. Humphreys,et al.  The paired-object affordance effect. , 2010, Journal of experimental psychology. Human perception and performance.

[141]  Roel M. Willems,et al.  The Function of Words: Distinct Neural Correlates for Words Denoting Differently Manipulable Objects , 2010, Journal of Cognitive Neuroscience.

[142]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[143]  S. Gallagher,et al.  The Phenomenological Mind: An Introduction to Philosophy of Mind and Cognitive Science , 2007 .

[144]  T. Ziemke,et al.  Theories and computational models of affordance and mirror systems: An integrative review , 2013, Neuroscience & Biobehavioral Reviews.

[145]  R. Lemon,et al.  What We Know Currently about Mirror Neurons , 2013, Current Biology.

[146]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[147]  Joan Y. Chiao,et al.  Neural re-use as a fundamental organizational principle of the brain . NEURAL RE-USE IN THE SOCIAL AND EMOTIONAL BRAIN , 2011 .

[148]  Daniel D. Hutto The Limits of Spectatorial Folk Psychology , 2004 .

[149]  Vittorio Gallese,et al.  Emotional and Social Behaviors Elicited by Electrical Stimulation of the Insula in the Macaque Monkey , 2011, Current Biology.