Prediction mechanisms in motor and auditory areas and their role in sound perception and language understanding

Is the meaning of an expected stimulus manifest in brain activity even before it appears? Although theories of predictive coding see anticipatory activity as crucial for the understanding of brain function, few studies have explored neurophysiologically manifest semantic predictions. Here, we report predictive negative-going potentials before the onset of action (i.e. whistle and hand clap) and non-action (i.e. pure tone, water drop) sounds. These prediction potentials (PP) indexed the meaning of action-related sounds. Dependent on the body-part-relationship of sound stimuli, neural sources were relatively more prominent in dorsal or ventral motor areas. In contrast, meaningless sounds (pure tones) activated predictive sources in temporal areas close to the auditory cortex; complex environmental sounds induced an anticipatory positivity broadly distributed over the scalp. We also found a systematic relationship between predictive activity and a Mismatch Negativity (MMN) like response to unexpected meaningful words which were presented as rare deviant stimuli amongst frequently repeated sounds. This deviant-elicited potential indexed semantic priming between action sounds and action-related words and semantic mismatch (prediction error). These results suggest a systematic link between perceptual/semantic prediction and matching mechanisms in the processing of sounds and words.

[1]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[2]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[3]  F. Pulvermüller,et al.  Language outside the focus of attention: The mismatch negativity as a tool for studying higher cognitive processes , 2006, Progress in Neurobiology.

[4]  M. Chait,et al.  Great expectations: Is there evidence for predictive coding in auditory cortex? , 2017, Neuroscience.

[5]  Aina Puce,et al.  Different categories of living and non-living sound-sources activate distinct cortical networks , 2009, NeuroImage.

[6]  Lüder Deecke,et al.  Brain potential changes in voluntary and passive movements in humans: readiness potential and reafferent potentials , 2016, Pflügers Archiv - European Journal of Physiology.

[7]  L. Fogassi,et al.  Audiovisual mirror neurons and action recognition , 2003, Experimental Brain Research.

[8]  Karl J. Friston,et al.  EEG and MEG Data Analysis in SPM8 , 2011, Comput. Intell. Neurosci..

[9]  W. Walter,et al.  Contingent Negative Variation : An Electric Sign of Sensori-Motor Association and Expectancy in the Human Brain , 1964, Nature.

[10]  Friedemann Pulvermüller,et al.  From sounds to words: A neurocomputational model of adaptation, inhibition and memory processes in auditory change detection , 2011, NeuroImage.

[11]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[12]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[13]  Friedemann Pulvermüller,et al.  Is the Sensorimotor Cortex Relevant for Speech Perception and Understanding? An Integrative Review , 2016, Front. Hum. Neurosci..

[14]  Friedemann Pulvermüller,et al.  Brain connections of words, perceptions and actions: A neurobiological model of spatio-temporal semantic activation in the human cortex , 2017, Neuropsychologia.

[15]  Friedemann Pulvermüller,et al.  Neuronal correlates of decisions to speak and act: Spontaneous emergence and dynamic topographies in a computational model of frontal and temporal areas , 2013, Brain and Language.

[16]  H. Helmholtz Ueber einige Gesetze der Vertheilung elektrischer Ströme in körperlichen Leitern mit Anwendung auf die thierisch‐elektrischen Versuche , 1853 .

[17]  F. Pulvermüller,et al.  Neurophysiological evidence for whole form retrieval of complex derived words: a mismatch negativity study , 2014, Front. Hum. Neurosci..

[18]  David Friedman,et al.  Brain potentials to sexually suggestive whistles show meaning modulates the mismatch negativity , 2005, Neuroreport.

[19]  R. Zatorre,et al.  Structure and function of auditory cortex: music and speech , 2002, Trends in Cognitive Sciences.

[20]  D. Mumford On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[21]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[22]  H. Kornhuber,et al.  Distribution of readiness potential, pre-motion positivity, and motor potential of the human cerebral cortex preceding voluntary finger movements , 2004, Experimental Brain Research.

[23]  G. Rizzolatti,et al.  Hearing Sounds, Understanding Actions: Action Representation in Mirror Neurons , 2002, Science.

[24]  Carles Escera,et al.  Attention capture by auditory significant stimuli: semantic analysis follows attention switching , 2003, The European journal of neuroscience.

[25]  S. Luck,et al.  The Oxford handbook of event-related potential components , 2011 .

[26]  Friedemann Pulvermüller,et al.  Behavioral / Cognitive Neural Correlates of Semantic Prediction and Resolution in Sentence Processing , 2017 .

[27]  S. Blakemore,et al.  Motor activation prior to observation of a predicted movement , 2004, Nature Neuroscience.

[28]  F. Pulvermüller,et al.  Somatotopic Semantic Priming and Prediction in the Motor System , 2016, Cerebral cortex.

[29]  W. Marslen-Wilson,et al.  Continuous uptake of acoustic cues in spoken word recognition , 1987, Perception & psychophysics.

[30]  Paavo Alku,et al.  Memory Traces for Words as Revealed by the Mismatch Negativity , 2001, NeuroImage.

[31]  F Pulvermüller,et al.  The sound of actions as reflected by mismatch negativity: rapid activation of cortical sensory–motor networks by sounds associated with finger and tongue movements , 2006, The European journal of neuroscience.

[32]  R. Näätänen The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function , 1990, Behavioral and Brain Sciences.

[33]  Elyse Sussman,et al.  The effect of stimulus context on the buildup to stream segregation , 2014, Front. Neurosci..

[34]  A. Dale,et al.  Human posterior auditory cortex gates novel sounds to consciousness. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Jeffrey R Binder,et al.  Human brain regions involved in recognizing environmental sounds. , 2004, Cerebral cortex.

[36]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[37]  H. Yabe,et al.  Temporal window of integration revealed by MMN to sound omission , 1997, Neuroreport.

[38]  Karl J. Friston,et al.  A Neurocomputational Model of the Mismatch Negativity , 2013, PLoS Comput. Biol..

[39]  R. Ilmoniemi,et al.  Language-specific phoneme representations revealed by electric and magnetic brain responses , 1997, Nature.

[40]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[41]  Karl J. Friston,et al.  Electromagnetic source reconstruction for group studies , 2008, NeuroImage.

[42]  A. Dale,et al.  Tonotopic organization in human auditory cortex revealed by progressions of frequency sensitivity. , 2004, Journal of neurophysiology.

[43]  W. Ziegler The Oxford Handbook Of Event Related Potential Components , 2016 .

[44]  J. Mäkelä,et al.  Spatiotemporal Dynamics of the Processing of Spoken Inflected and Derived Words: A Combined EEG and MEG Study , 2011, Front. Hum. Neurosci..

[45]  Friedemann Pulvermüller,et al.  Early MEG Activation Dynamics in the Left Temporal and Inferior Frontal Cortex Reflect Semantic Context Integration , 2007, Journal of Cognitive Neuroscience.

[46]  Jacob Cohen Measurement Educational and Psychological Educational and Psychological Measurement Eta-squared and Partial Eta-squared in Fixed Factor Anova Designs Educational and Psychological Measurement Additional Services and Information For , 2022 .

[47]  William D. Marslen-Wilson,et al.  Tracking speech comprehension in space and time , 2006, NeuroImage.

[48]  K. Alho,et al.  Separate Time Behaviors of the Temporal and Frontal Mismatch Negativity Sources , 2000, NeuroImage.

[49]  Karl J. Friston,et al.  Multiple sparse priors for the M/EEG inverse problem , 2008, NeuroImage.

[50]  Y. Shtyrov,et al.  Automatic ultrarapid activation and inhibition of cortical motor systems in spoken word comprehension , 2014, Proceedings of the National Academy of Sciences.

[51]  Friedemann Pulvermüller,et al.  Brain Signatures of Meaning Access in Action Word Recognition , 2005, Journal of Cognitive Neuroscience.

[52]  H. Tiitinen,et al.  The MMN is a derivative of the auditory N100 response. , 2004, Neurology & clinical neurophysiology : NCN.

[53]  Karl J. Friston,et al.  The functional anatomy of the MMN: A DCM study of the roving paradigm , 2008, NeuroImage.

[54]  L. Fadiga,et al.  Active perception: sensorimotor circuits as a cortical basis for language , 2010, Nature Reviews Neuroscience.

[55]  C. Heyes,et al.  Mirror neurons: from origin to function. , 2014, The Behavioral and brain sciences.

[56]  Paul J. Laurienti,et al.  An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets , 2003, NeuroImage.

[57]  R. Näätänen,et al.  The mismatch negativity (MMN) in basic research of central auditory processing: A review , 2007, Clinical Neurophysiology.

[58]  Christina M. Krause,et al.  Early and Late Mismatch Negativity Elicited by Words and Speech-Like Stimuli in Children , 2001, Brain and Language.

[59]  Friedemann Pulvermüller,et al.  Understanding in an instant: Neurophysiological evidence for mechanistic language circuits in the brain , 2009, Brain and Language.

[60]  R. Näätänen,et al.  Early selective-attention effect on evoked potential reinterpreted. , 1978, Acta psychologica.

[61]  Daphne Manoussaki,et al.  The influence of cochlear shape on low-frequency hearing , 2008, Proceedings of the National Academy of Sciences.