Synthetic event-related potentials: A computational bridge between neurolinguistic models and experiments

Our previous work developed Synthetic Brain Imaging to link neural and schema network models of cognition and behavior to PET and fMRI studies of brain function. We here extend this approach to Synthetic Event-Related Potentials (Synthetic ERP). Although the method is of general applicability, we focus on ERP correlates of language processing in the human brain. The method has two components: Phase 1: To generate cortical electro-magnetic source activity from neural or schema network models; and Phase 2: To generate known neurolinguistic ERP data (ERP scalp voltage topographies and waveforms) from putative cortical source distributions and activities within a realistic anatomical model of the human brain and head. To illustrate the challenges of Phase 2 of the methodology, spatiotemporal information from Friederici's 2002 model of auditory language comprehension was used to define cortical regions and time courses of activation for implementation within a forward model of ERP data. The cortical regions from the 2002 model were modeled using atlas-based masks overlaid on the MNI high definition single subject cortical mesh. The electromagnetic contribution of each region was modeled using current dipoles whose position and orientation were constrained by the cortical geometry. In linking neural network computation via EEG forward modeling to empirical results in neurolinguistics, we emphasize the need for neural network models to link their architecture to geometrically sound models of the cortical surface, and the need for conceptual models to refine and adopt brain-atlas based approaches to allow precise brain anchoring of their modules. The detailed analysis of Phase 2 sets the stage for a brief introduction to Phase 1 of the program, including the case for a schema-theoretic approach to language production and perception presented in detail elsewhere. Unlike Dynamic Causal Modeling (DCM) and Bojak's mean field model, Synthetic ERP builds on models of networks that mediate the relation between the brain's inputs, outputs, and internal states in executing a specific task. The neural networks used for Synthetic ERP must include neuroanatomically realistic placement and orientation of the cortical pyramidal neurons. These constraints pose exciting challenges for future work in neural network modeling that is applicable to systems and cognitive neuroscience.

[1]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[2]  C. Leonard,et al.  Normal variation in the frequency and location of human auditory cortex landmarks. Heschl's gyrus: where is it? , 1998, Cerebral cortex.

[3]  E. Bullmore,et al.  Common and Distinct Neural Substrates for Pragmatic, Semantic, and Syntactic Processing of Spoken Sentences: An fMRI Study , 2000, Journal of Cognitive Neuroscience.

[4]  G. Curio,et al.  The Human Thalamus Processes Syntactic and Semantic Language Violations , 2008, Neuron.

[5]  D. L. Schomer,et al.  Niedermeyer's Electroencephalography: Basic Principles, Clinical Applications, and Related Fields , 2012 .

[6]  Victor R. Lesser,et al.  The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty , 1980, CSUR.

[7]  Karl J. Friston,et al.  The Dynamic Brain: From Spiking Neurons to Neural Masses and Cortical Fields , 2008, PLoS Comput. Biol..

[8]  P. Nunez,et al.  On the Relationship of Synaptic Activity to Macroscopic Measurements: Does Co-Registration of EEG with fMRI Make Sense? , 2004, Brain Topography.

[9]  B. Horwitz,et al.  Integrating electrophysiological and anatomical experimental data to create a large-scale model that simulates a delayed match-to-sample human brain imaging study. , 1998, Cerebral cortex.

[10]  Sharlene D. Newman,et al.  Off‐line sentence processing: What is involved in answering a comprehension probe? , 2009, Human brain mapping.

[11]  Michael A. Arbib,et al.  From Schema Theory To Language , 1987 .

[12]  Moritz Grosse-Wentrup,et al.  Multisubject Learning for Common Spatial Patterns in Motor-Imagery BCI , 2011, Comput. Intell. Neurosci..

[13]  Peter Ford Dominey,et al.  A Neurolinguistic Model of Grammatical Construction Processing , 2006, Journal of Cognitive Neuroscience.

[14]  Peter Ford Dominey,et al.  Neural network processing of natural language: II. Towards a unified model of corticostriatal function in learning sentence comprehension and non-linguistic sequencing , 2009, Brain and Language.

[15]  Nikos Makris,et al.  Automatically parcellating the human cerebral cortex. , 2004, Cerebral cortex.

[16]  Katrin Amunts,et al.  Cortical Folding Patterns and Predicting Cytoarchitecture , 2007, Cerebral cortex.

[17]  Bart Vanrumste,et al.  EEG/MEG Source Imaging: Methods, Challenges, and Open Issues , 2009, Comput. Intell. Neurosci..

[18]  Friedemann Pulvermüller,et al.  A neuroanatomically grounded Hebbian-learning model of attention–language interactions in the human brain , 2008, The European journal of neuroscience.

[19]  Gregory Hickok,et al.  The functional neuroanatomy of language. , 2009, Physics of life reviews.

[20]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[21]  A. Friederici,et al.  Differential task effects on semantic and syntactic processes as revealed by ERPs. , 2002, Brain research. Cognitive brain research.

[22]  Anders M. Dale,et al.  Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature , 2010, NeuroImage.

[23]  Scott T. Grafton,et al.  Dorsal premotor cortex and conditional movement selection: A PET functional mapping study. , 1998, Journal of neurophysiology.

[24]  Alain Trouvé,et al.  Diffeomorphic Brain Registration Under Exhaustive Sulcal Constraints , 2011, IEEE Transactions on Medical Imaging.

[25]  Karl J. Friston,et al.  A neural mass model for MEG/EEG: coupling and neuronal dynamics , 2003, NeuroImage.

[26]  D. Heeger,et al.  In this issue , 2002, Nature Reviews Drug Discovery.

[27]  Angela D. Friederici,et al.  Pathways to language: fiber tracts in the human brain , 2009, Trends in Cognitive Sciences.

[28]  Bernard Mazoyer,et al.  Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence processing , 2006, NeuroImage.

[29]  Michael A. Arbib,et al.  Describing visual scenes: Towards a neurolinguistics based on construction grammar , 2008, Brain Research.

[30]  F. Aboitiz Gestures, Vocalizations, and Memory in Language Origins , 2011, Front. Evol. Neurosci..

[31]  F. T. Husain,et al.  Relating neuronal dynamics for auditory object processing to neuroimaging activity: a computational modeling and an fMRI study , 2004, NeuroImage.

[32]  T. Picton,et al.  The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. , 1987, Psychophysiology.

[33]  Michael A. Arbib,et al.  Mirror system activity for action and language is embedded in the integration of dorsal and ventral pathways , 2010, Brain and Language.

[34]  Stephen G. Waxman,et al.  The conduction properties of axons in central white matter , 1977, Progress in Neurobiology.

[35]  A. Friederici The brain basis of language processing: from structure to function. , 2011, Physiological reviews.

[36]  Daniel Bullock,et al.  Neural Representations and Mechanisms for the Performance of Simple Speech Sequences , 2010, Journal of Cognitive Neuroscience.

[37]  D. Louis Collins,et al.  Brain templates and atlases , 2012, NeuroImage.

[38]  A. Scheibel,et al.  Fiber composition of the human corpus callosum , 1992, Brain Research.

[39]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[40]  Scott T. Grafton,et al.  Synthetic PET: Analyzing large‐scale properties of neural networks , 1994 .

[41]  G. Dell,et al.  Becoming syntactic. , 2006, Psychological review.

[42]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[43]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[44]  Michael A. Arbib,et al.  Synthetic brain imaging: grasping, mirror neurons and imitation , 2000, Neural Networks.

[45]  Michael A. Arbib,et al.  Language evolution: neural homologies and neuroinformatics , 2003, Neural Networks.

[46]  Fernando Lopes da Silva,et al.  Biophysical Aspects of EEG and MEG , 2022 .

[47]  A. Friederici,et al.  The brain basis of syntactic processes: functional imaging and lesion studies , 2003, NeuroImage.

[48]  Valer Jurcak,et al.  10/20, 10/10, and 10/5 systems revisited: Their validity as relative head-surface-based positioning systems , 2007, NeuroImage.

[49]  Théodore Papadopoulo,et al.  Forward Field Computation with OpenMEEG , 2011, Comput. Intell. Neurosci..

[50]  M. García-Fiñana,et al.  Sulcal variability, stereological measurement and asymmetry of Broca's area on MR images , 2007, Journal of anatomy.

[51]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[52]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[53]  H A Whitaker,et al.  ANATOMIC VARIATIONS IN THE CORTEX: INDIVIDUAL DIFFERENCES AND THE PROBLEM OF THE LOCALIZATION OF LANGUAGE FUNCTIONS , 1976, Annals of the New York Academy of Sciences.

[54]  Karl J. Friston,et al.  Systematic Regularization of Linear Inverse Solutions of the EEG Source Localization Problem , 2002, NeuroImage.

[55]  R. Kötter,et al.  Connecting Mean Field Models of Neural Activity to EEG and fMRI Data , 2010, Brain Topography.

[56]  Nelson J. Trujillo-Barreto,et al.  Realistically Coupled Neural Mass Models Can Generate EEG Rhythms , 2007, Neural Computation.

[57]  Michael A. Arbib,et al.  Vision and Action in the Language-Ready Brain: From Mirror Neurons to SemRep , 2007, BVAI.

[58]  T. Deacon,et al.  Monkey homologues of language areas: computing the ambiguities , 2004, Trends in Cognitive Sciences.

[59]  P. Morosan,et al.  Broca's Region: Novel Organizational Principles and Multiple Receptor Mapping , 2010, PLoS biology.

[60]  E. Schröger,et al.  Suppression of the auditory N1 event-related potential component with unpredictable self-initiated tones: evidence for internal forward models with dynamic stimulation. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[61]  G. H. Bishop,et al.  THE SIZE OF NERVE FIBERS SUPPLYING CEREBRAL CORTEX. , 1964, Experimental neurology.

[62]  Francisco Aboitiz,et al.  The Phonological Loop , 2010, Current Anthropology.

[63]  Fernando Lopes da Silva,et al.  Comprar Niedermeyer's Electroencephalography, 6/e (Basic Principles, Clinical Applications, and Related Fields ) | Fernando Lopes Da Silva | 9780781789424 | Lippincott Williams & Wilkins , 2010 .

[64]  J. Sarvas Basic mathematical and electromagnetic concepts of the biomagnetic inverse problem. , 1987, Physics in medicine and biology.

[65]  M. Young,et al.  Advanced database methodology for the Collation of Connectivity data on the Macaque brain (CoCoMac). , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[66]  Steven Robbins,et al.  An unbiased iterative group registration template for cortical surface analysis , 2007, NeuroImage.

[67]  Mathew P. Dafilis,et al.  A spatially continuous mean field theory of electrocortical activity , 2002, Network.

[68]  Arnaud Cachia,et al.  In-vivo measurement of cortical morphology: means and meanings. , 2010, Current opinion in neurology.

[69]  Michael A. Arbib,et al.  A Model of Primate Visual-Motor Conditional Learning , 1992, Adapt. Behav..

[70]  Scott T. Grafton,et al.  Synthetic PET imaging for grasping: from primate Neurophysiology to human behavior , 2003 .

[71]  Bart Vanrumste,et al.  Journal of Neuroengineering and Rehabilitation Open Access Review on Solving the Inverse Problem in Eeg Source Analysis , 2022 .

[72]  H. Lüders,et al.  Functional connectivity in the human language system: a cortico-cortical evoked potential study. , 2004, Brain : a journal of neurology.

[73]  J. Fermaglich Electric Fields of the Brain: The Neurophysics of EEG , 1982 .

[74]  Gerard Kempen,et al.  An Incremental Procedural Grammar for Sentence Formulation , 1987, Cogn. Sci..

[75]  Thom F. Oostendorp,et al.  The conductivity of the human skull: results of in vivo and in vitro measurements , 2000, IEEE Transactions on Biomedical Engineering.

[76]  Cornelius Weiller,et al.  How the ventral pathway got lost – And what its recovery might mean , 2011, Brain and Language.

[77]  Jinyong Lee,et al.  Template Construction Grammar: From Visual Scene Description to Language Comprehension and Agrammatism , 2013, Neuroinformatics.

[78]  Katrin Amunts,et al.  Architecture and organizational principles of Broca's region , 2012, Trends in Cognitive Sciences.

[79]  Walter Ritter,et al.  6 On Relating Event-Related Potential Components to Stages of Information Processing* , 1983 .

[80]  K. Lehnertz,et al.  Tonotopic organization of the human auditory cortex revealed by transient auditory evoked magnetic fields. , 1988, Electroencephalography and clinical neurophysiology.

[81]  M. Arbib,et al.  Neurolinguistics must be computational , 1979 .

[82]  S. Curry,et al.  The form and distribution of auditory evoked potentials and CNVs when stimuli and responses are lateralized. , 1980, Progress in brain research.

[83]  E. Rolls,et al.  What and Where in Visual Working Memory: A Computational Neurodynamical Perspective for Integrating fMRI and Single-Neuron Data , 2004, Journal of Cognitive Neuroscience.

[84]  A. Friederici The cortical language circuit: from auditory perception to sentence comprehension , 2012, Trends in Cognitive Sciences.

[85]  Ben H. Jansen,et al.  Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns , 1995, Biological Cybernetics.

[86]  A. Nakamura,et al.  Localizing the distributed language network responsible for the N400 measured by MEG during auditory sentence processing , 2006, Brain Research.

[87]  Victor Lesser,et al.  Organization of the Hearsay II speech understanding system , 1975 .

[88]  Karl J. Friston,et al.  Modelling event-related responses in the brain , 2005, NeuroImage.

[89]  G. Dell,et al.  Lexical access in aphasic and nonaphasic speakers. , 1997, Psychological review.

[90]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[91]  A. Friederici,et al.  Dynamic Causal Modeling of Subcortical Connectivity of Language , 2011, The Journal of Neuroscience.

[92]  Marcela Perrone-Bertolotti,et al.  Dynamic Causal Modeling of Spatiotemporal Integration of Phonological and Semantic Processes: An Electroencephalographic Study , 2012, The Journal of Neuroscience.

[93]  Karl J. Friston,et al.  Dynamic causal modeling of evoked responses in EEG and MEG , 2006, NeuroImage.

[94]  M. Arbib Aphasia, apraxia and the evolution of the language-ready brain , 2006 .

[95]  M. Murray,et al.  EEG source imaging , 2004, Clinical Neurophysiology.

[96]  F. Chang Symbolically speaking: a connectionist model of sentence production , 2002 .

[97]  M. Scherg,et al.  A Source Analysis of the Late Human Auditory Evoked Potentials , 1989, Journal of Cognitive Neuroscience.

[98]  M. Penke,et al.  How the brain processes complex words: an ERP-study of German verb inflections , 1997 .

[99]  Barry Horwitz,et al.  A model of working memory: bridging the gap between electrophysiology and human brain imaging , 2000, Neural Networks.

[100]  Kristen A. Lindgren,et al.  Effective and Structural Connectivity in the Human Auditory Cortex , 2008, The Journal of Neuroscience.

[101]  David C. Van Essen,et al.  A Population-Average, Landmark- and Surface-based (PALS) atlas of human cerebral cortex , 2005, NeuroImage.

[102]  A. Friederici,et al.  Electrophysiological Evidence for Two Steps in Syntactic Analysis: Early Automatic and Late Controlled Processes , 1999, Journal of Cognitive Neuroscience.

[103]  G. V. Simpson,et al.  A test of brain electrical source analysis (BESA): a simulation study. , 1994, Electroencephalography and clinical neurophysiology.

[104]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[105]  Katrin Amunts,et al.  Broca’s area: Nomenclature, anatomy, typology and asymmetry , 2009, Brain and Language.

[106]  D. Lehmann,et al.  Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. , 1994, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[107]  Bart Vanrumste,et al.  Review on solving the forward problem in EEG source analysis , 2007, Journal of NeuroEngineering and Rehabilitation.

[108]  Ernst Fernando Lopes Da Silva Niedermeyer,et al.  Electroencephalography, basic principles, clinical applications, and related fields , 1982 .

[109]  Michael A. Arbib,et al.  Précis of How the brain got language: The Mirror System Hypothesis , 2012, Language and Cognition.

[111]  F Lhermitte,et al.  Phonemic paraphasias: linguistic structures and tentative hypothesis. , 1969, Cortex; a journal devoted to the study of the nervous system and behavior.

[112]  P. Holcomb,et al.  Event-related brain potentials elicited by syntactic anomaly , 1992 .

[113]  P. Hagoort On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[114]  J. Rauschecker,et al.  Phoneme and word recognition in the auditory ventral stream , 2012, Proceedings of the National Academy of Sciences.

[115]  H. Helmholtz Ueber einige Gesetze der Vertheilung elektrischer Ströme in körperlichen Leitern mit Anwendung auf die thierisch‐elektrischen Versuche , 1853 .

[116]  Peter Ford Dominey,et al.  A cortico-subcortical model for generation of spatially accurate sequential saccades. , 1992, Cerebral cortex.

[117]  Peter Ford Dominey,et al.  A model of the cerebellum in adaptive control of saccadic gain , 1996, Biological cybernetics.

[118]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[119]  Arthur W. Toga,et al.  A Probabilistic Atlas of the Human Brain: Theory and Rationale for Its Development The International Consortium for Brain Mapping (ICBM) , 1995, NeuroImage.

[120]  Peter Ford Dominey,et al.  A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences , 1995, Journal of Cognitive Neuroscience.