Semantic Probing: Feasibility of using sequential probes to decode what is on a user’s mind

In this paper, we investigate the feasibility of using multiple sequential probe words to decode their relatedness to an active semantic concept on a user’s mind from the respective electrophysiological brain responses. If feasible, this relatedness information could be used by a Brain Computer Interface to infer that semantic concept, by integrating the knowledge of the relationship between the multiple probe words and the ‘unknown’ target. Such a BCI can take advantage of the N400: an event related potential that is sensitive to semantic content of a stimulus in relation to an established semantic context. However, it is unknown whether the N400 is suited for the multiple probing paradigm we propose, as other intervening words might distract from the established context (i.e., the target word). We perform an experiment in which we present up to ten words after an initial target word, and find no attenuation of the strength of the N400 in grand average ERPs and no decrease in classification accuracy for probes occurring later in the sequences. These results lay the groundwork for the development of a BCI that infers the concept on a user’s mind through repeated probing.

[1]  Peter Desain,et al.  Introducing the tactile speller: an ERP-based brain–computer interface for communication , 2012, Journal of neural engineering.

[2]  S. Luck,et al.  How inappropriate high-pass filters can produce artifactual effects and incorrect conclusions in ERP studies of language and cognition. , 2015, Psychophysiology.

[3]  Peter Desain,et al.  Detecting Semantic Priming at the Single-Trial Level , 2013, PloS one.

[4]  Kimitaka Kaga,et al.  Auditory lexical-semantic processing impairments in aphasic patients reflected in event-related potentials (N400). , 2003, Auris, nasus, larynx.

[5]  Jason Farquhar,et al.  Interactions Between Pre-Processing and Classification Methods for Event-Related-Potential Classification , 2012, Neuroinformatics.

[6]  H. Neville,et al.  Language and , 2019, Adventure Diffusion.

[7]  Peter Desain,et al.  Broad-Band Visually Evoked Potentials: Re(con)volution in Brain-Computer Interfacing , 2015, PloS one.

[8]  J. Schoffelen,et al.  Beta oscillations relate to the N400m during language comprehension , 2012, Human brain mapping.

[9]  J. Wolpaw,et al.  A practical, intuitive brain–computer interface for communicating ‘yes’ or ‘no’ by listening , 2014, Journal of neural engineering.

[10]  Benjamin Blankertz,et al.  A gaze independent spelling based on rapid serial visual presentation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Matti Laine,et al.  Anomia: Theoretical and Clinical Aspects , 2006 .

[12]  Jason Farquhar,et al.  Towards a Communication Brain Computer Interface Based on Semantic Relations , 2014, PloS one.

[13]  Colin M. Brown,et al.  ERP effects of listening to speech: semantic ERP effects , 2000, Neuropsychologia.

[14]  J. Wolpaw,et al.  Answering questions with an electroencephalogram-based brain-computer interface. , 1998, Archives of physical medicine and rehabilitation.

[15]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[16]  B Blankertz,et al.  Real-time inference of word relevance from electroencephalogram and eye gaze , 2017, Journal of neural engineering.

[17]  Colin M. Brown,et al.  Lexical-semantic event-related potential effects in patients with left hemisphere lesions and aphasia, and patients with right hemisphere lesions without aphasia. , 1996, Brain : a journal of neurology.

[18]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[19]  W. Ritter,et al.  The relationship between N2 and N400: scalp distribution, stimulus probability, and task relevance. , 1991, Psychophysiology.

[20]  Bin Xia,et al.  Brain–Computer Interface–Based Communication in the Completely Locked-In State , 2017, GBCIC.

[21]  M. Rugg.  Event-related brain potentials dissociate repetition effects of high-and low-frequency words , 1990, Memory & cognition.

[22]  D Deacon,et al.  How automatically is meaning accessed: a review of the effects of attention on semantic processing. , 2000, Frontiers in bioscience : a journal and virtual library.

[23]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[24]  F. Perrin,et al.  Spherical splines for scalp potential and current density mapping. , 1989, Electroencephalography and clinical neurophysiology.

[25]  M. Kutas,et al.  The Search for Common Sense: An Electrophysiological Study of the Comprehension of Words and Pictures in Reading , 1996, Journal of Cognitive Neuroscience.

[26]  G. Gratton.  Dealing with artifacts: The EOG contamination of the event-related brain potential , 1998 .

[27]  E. Maris.  Statistical testing in electrophysiological studies. , 2012, Psychophysiology.

[28]  Gert Storms,et al.  Word associations: Norms for 1,424 Dutch words in a continuous task , 2008, Behavior research methods.

[29]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[30]  P. Holcomb,et al.  An electrophysiological investigation of semantic priming with pictures of real objects. , 1999, Psychophysiology.

[31]  Jeffrey G Malins,et al.  Neuroimage: Clinical the Reliability of the N400 in Single Subjects: Implications for Patients with Disorders of Consciousness , 2022 .

[32]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..