Only time will tell – why temporal information is essential for our neuroscientific understanding of semantics

Theoretical developments about the nature of semantic representations and processes should be accompanied by a discussion of how these theories can be validated on the basis of empirical data. Here, I elaborate on the link between theory and empirical research, highlighting the need for temporal information in order to distinguish fundamental aspects of semantics. The generic point that fast cognitive processes demand fast measurement techniques has been made many times before, although arguably more often in the psychophysiological community than in the metabolic neuroimaging community. Many reviews on the neuroscience of semantics mostly or even exclusively focus on metabolic neuroimaging data. Following an analysis of semantics in terms of the representations and processes involved, I argue that fundamental theoretical debates about the neuroscience of semantics can only be concluded on the basis of data with sufficient temporal resolution. Any “semantic effect” may result from a conflation of long-term memory representations, retrieval and working memory processes, mental imagery, and episodic memory. This poses challenges for all neuroimaging modalities, but especially for those with low temporal resolution. It also throws doubt on the usefulness of contrasts between meaningful and meaningless stimuli, which may differ on a number of semantic and non-semantic dimensions. I will discuss the consequences of this analysis for research on the role of convergence zones or hubs and distributed modal brain networks, top-down modulation of task and context as well as interactivity between levels of the processing hierarchy, for example in the framework of predictive coding.

[1]  J. Deutsch,et al.  Militarized youths in western Côte d’Ivoire: local processes of mobilization, demobilization, and related humanitarian interventions (2002-2007) , 2011 .

[2]  J. Deutsch,et al.  Attention: Some theoretical considerations. , 1963 .

[3]  A. Treisman,et al.  Is selective attention selective perception or selective response? A further test. , 1969, Journal of experimental psychology.

[4]  John R. Anderson Arguments concerning representations for mental imagery. , 1978 .

[5]  T Shallice,et al.  Semantic access dyslexia. , 1979, Brain : a journal of neurology.

[6]  John R. Searle,et al.  Minds, brains, and programs , 1980, Behavioral and Brain Sciences.

[7]  D. Balota,et al.  Are lexical decisions a good measure of lexical access? The role of word frequency in the neglected decision stage. , 1984, Journal of experimental psychology. Human perception and performance.

[8]  A. Damasio Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition , 1989, Cognition.

[9]  A. Caramazza,et al.  On the distinction between deficits of access and deficits of storage : a question of theory , 1993 .

[10]  M. Farah,et al.  Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Karl J. Friston,et al.  A neuromodulatory role for the human amygdala in processing emotional facial expressions. , 1998, Brain : a journal of neurology.

[12]  R. Buckner,et al.  Human Brain Mapping 6:373–377(1998) � Event-Related fMRI and the Hemodynamic Response , 2022 .

[13]  F. Pulvermüller Words in the brain's language , 1999, Behavioral and Brain Sciences.

[14]  M. Jeannerod,et al.  Mental imaging of motor activity in humans , 1999, Current Opinion in Neurobiology.

[15]  Margot J. Taylor,et al.  Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. , 2000, Psychophysiology.

[16]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[17]  C. Price The anatomy of language: contributions from functional neuroimaging , 2000, Journal of anatomy.

[18]  J. Bullier Integrated model of visual processing , 2001, Brain Research Reviews.

[19]  Refractor Vision , 2000, The Lancet.

[20]  Andy C. H. Lee,et al.  Regional brain activations differ for semantic features but not categories , 2002, Neuroreport.

[21]  A. Baddeley Working memory and language: an overview. , 2003, Journal of communication disorders.

[22]  A. Mechelli,et al.  Neuroimaging Studies of Word and Pseudoword Reading: Consistencies, Inconsistencies, and Limitations , 2003, Journal of Cognitive Neuroscience.

[23]  G Jobard,et al.  Evaluation of the dual route theory of reading: a metanalysis of 35 neuroimaging studies , 2003, NeuroImage.

[24]  S. Hochstein,et al.  The reverse hierarchy theory of visual perceptual learning , 2004, Trends in Cognitive Sciences.

[25]  James L. McClelland,et al.  Structure and deterioration of semantic memory: a neuropsychological and computational investigation. , 2004, Psychological review.

[26]  Roger Ratcliff,et al.  A diffusion model account of the lexical decision task. , 2004, Psychological review.

[27]  Cathy J. Price,et al.  Fusiform Activation to Animals is Driven by the Process, Not the Stimulus , 2005, Journal of Cognitive Neuroscience.

[28]  Michael S. Beauchamp,et al.  Automatic Priming of Semantically Related Words Reduces Activity in the Fusiform Gyrus , 2005, Journal of Cognitive Neuroscience.

[29]  R. Henson What can Functional Neuroimaging Tell the Experimental Psychologist? , 2005, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[30]  S. Kosslyn Mental images and the Brain , 2005, Cognitive neuropsychology.

[31]  Charles D. Smith,et al.  Dissociation of Automatic and Strategic Lexical-Semantics: Functional Magnetic Resonance Imaging Evidence for Differing Roles of Multiple Frontotemporal Regions , 2006, The Journal of Neuroscience.

[32]  E. Halgren,et al.  Top-down facilitation of visual recognition. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[33]  M. Page,et al.  What Can't Functional Neuroimaging Tell the Cognitive Psychologist? , 2006, Cortex.

[34]  William D. Marslen-Wilson,et al.  The time course of visual word recognition as revealed by linear regression analysis of ERP data , 2006, NeuroImage.

[35]  M. Berger,et al.  High Gamma Power Is Phase-Locked to Theta Oscillations in Human Neocortex , 2006, Science.

[36]  S. Harnad Symbol grounding problem , 1991, Scholarpedia.

[37]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[38]  K. Watkins,et al.  Stimulating language: insights from TMS. , 2007, Brain : a journal of neurology.

[39]  David A. Balota,et al.  Attentional Control and Flexible Lexical Processing : Explorations of the Magic Moment of Word Recognition , 2007 .

[40]  Matthew H. Davis,et al.  Imagery or meaning? Evidence for a semantic origin of category-specific brain activity in metabolic imaging , 2008, The European journal of neuroscience.

[41]  F. Pulvermüller,et al.  The time course of action and action-word comprehension in the human brain as revealed by neurophysiology , 2008, Journal of Physiology-Paris.

[42]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[43]  Bradford Z. Mahon,et al.  A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content , 2008, Journal of Physiology-Paris.

[44]  Dietrich Lehmann,et al.  EEG microstates , 2009, Scholarpedia.

[45]  A. Damasio,et al.  Convergence and divergence in a neural architecture for recognition and memory , 2009, Trends in Neurosciences.

[46]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[47]  Harold Bekkering,et al.  Context Effects in Embodied Lexical-Semantic Processing , 2010, Front. Psychology.

[48]  Elizabeth Jefferies,et al.  Semantic Processing in the Anterior Temporal Lobes: A Meta-analysis of the Functional Neuroimaging Literature , 2010, Journal of Cognitive Neuroscience.

[49]  A. Engel,et al.  Beta-band oscillations—signalling the status quo? , 2010, Current Opinion in Neurobiology.

[50]  Daniel Casasanto,et al.  Neural Dissociations between Action Verb Understanding and Motor Imagery , 2010, Journal of Cognitive Neuroscience.

[51]  Rutvik H. Desai,et al.  The neurobiology of semantic memory , 2011, Trends in Cognitive Sciences.

[52]  P. Hoffman,et al.  Remembering ‘zeal’ but not ‘thing’: Reverse frequency effects as a consequence of deregulated semantic processing , 2011, Neuropsychologia.

[53]  Ben D. Amsel Tracking real-time neural activation of conceptual knowledge using single-trial event-related potentials , 2011, Neuropsychologia.

[54]  Karl J. Friston,et al.  Effective connectivity: Influence, causality and biophysical modeling , 2011, NeuroImage.

[55]  A. Engel,et al.  Spectral fingerprints of large-scale neuronal interactions , 2012, Nature Reviews Neuroscience.

[56]  Y. Chen,et al.  The time-course of single-word reading: Evidence from fast behavioral and brain responses , 2012, NeuroImage.

[57]  Krish D. Singh,et al.  Which “neural activity” do you mean? fMRI, MEG, oscillations and neurotransmitters , 2012, NeuroImage.

[58]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[59]  Elizabeth Jefferies,et al.  Executive Semantic Processing Is Underpinned by a Large-scale Neural Network: Revealing the Contribution of Left Prefrontal, Posterior Temporal, and Parietal Cortex to Controlled Retrieval and Selection Using TMS , 2012, Journal of Cognitive Neuroscience.

[60]  M. Carandini From circuits to behavior: a bridge too far? , 2012, Nature Neuroscience.

[61]  F. Pulvermüller How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics , 2013, Trends in Cognitive Sciences.

[62]  Friedemann Pulvermüller,et al.  Moving the hands and feet specifically impairs working memory for arm- and leg-related action words , 2013, Cortex.

[63]  Matthew H. Davis,et al.  Can cognitive models explain brain activation during word and pseudoword reading? A meta-analysis of 36 neuroimaging studies. , 2013, Psychological bulletin.

[64]  Joachim Gross,et al.  Good practice for conducting and reporting MEG research , 2013, NeuroImage.

[65]  O. Hauk,et al.  The Body of Evidence: What Can Neuroscience Tell Us about Embodied Semantics? , 2013, Front. Psychology.

[66]  Marta Kutas,et al.  Alive and Grasping: Stable and Rapid Semantic Access to an Object Category but Not Object Graspability , 2022 .

[67]  F. Pulvermüller,et al.  Task modulation of brain responses in visual word recognition as studied using EEG/MEG and fMRI , 2013, Front. Hum. Neurosci..

[68]  W. Penny,et al.  Reading Front to Back: MEG Evidence for Early Feedback Effects During Word Recognition , 2012, Cerebral cortex.

[69]  Blair C. Armstrong,et al.  The what, when, where, and how of visual word recognition , 2014, Trends in Cognitive Sciences.

[70]  David Poeppel,et al.  Towards a computational(ist) neurobiology of language: correlational, integrated and explanatory neurolinguistics , 2015, Language, cognition and neuroscience.

[71]  Friedemann Pulvermüller,et al.  Early Visual Word Processing Is Flexible: Evidence from Spatiotemporal Brain Dynamics , 2015, Journal of Cognitive Neuroscience.

[72]  Bradford Z. Mahon,et al.  What is embodied about cognition? , 2015, Language, cognition and neuroscience.

[73]  Jonathan Grainger,et al.  Seeing the Same Words Differently: The Time Course of Automaticity and Top–Down Intention in Reading , 2015, Journal of Cognitive Neuroscience.

[74]  Benjamin Naumann,et al.  Mental Representations A Dual Coding Approach , 2016 .