Implementation of an Online Auditory Attention Detection Model with Electroencephalography in a Dichotomous Listening Experiment

Auditory attention detection (AAD) is the tracking of a sound source to which a listener is attending based on neural signals. Despite expectation for the applicability of AAD in real-life, most AAD research has been conducted on recorded electroencephalograms (EEGs), which is far from online implementation. In the present study, we attempted to propose an online AAD model and to implement it on a streaming EEG. The proposed model was devised by introducing a sliding window into the linear decoder model and was simulated using two datasets obtained from separate experiments to evaluate the feasibility. After simulation, the online model was constructed and evaluated based on the streaming EEG of an individual, acquired during a dichotomous listening experiment. Our model was able to detect the transient direction of a participant’s attention on the order of one second during the experiment and showed up to 70% average detection accuracy. We expect that the proposed online model could be applied to develop adaptive hearing aids or neurofeedback training for auditory attention and speech perception.

[1]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[2]  Birger Kollmeier,et al.  Machine learning for decoding listeners’ attention from electroencephalography evoked by continuous speech , 2020, The European journal of neuroscience.

[3]  K. Spencer,et al.  Poststimulus EEG spectral analysis and P300: attention, task, and probability. , 1999, Psychophysiology.

[4]  E. C. Cmm,et al.  on the Recognition of Speech, with , 2008 .

[5]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[6]  Maarten De Vos,et al.  Decoding the attended speech stream with multi-channel EEG: implications for online, daily-life applications , 2015, Journal of neural engineering.

[7]  Antje S. Meyer,et al.  Linguistic Structure and Meaning Organize Neural Oscillations into a Content-Specific Hierarchy , 2020, The Journal of Neuroscience.

[8]  David Poeppel,et al.  Speech rhythms and their neural foundations , 2020, Nature Reviews Neuroscience.

[9]  V. Jousmäki,et al.  The pace of prosodic phrasing couples the listener's cortex to the reader's voice , 2013, Human brain mapping.

[10]  Lenny A. Varghese,et al.  Quantifying attentional modulation of auditory-evoked cortical responses from single-trial electroencephalography , 2013, Front. Hum. Neurosci..

[11]  Jan Wouters,et al.  Speech Intelligibility Predicted from Neural Entrainment of the Speech Envelope , 2018, bioRxiv.

[12]  Edmund C. Lalor,et al.  The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli , 2016, Front. Hum. Neurosci..

[13]  C. Kayser,et al.  Neural Entrainment and Attentional Selection in the Listening Brain , 2019, Trends in Cognitive Sciences.

[14]  J. Gruzelier.  EEG-neurofeedback for optimising performance. I: A review of cognitive and affective outcome in healthy participants , 2014, Neuroscience & Biobehavioral Reviews.

[15]  Jonathan Z. Simon,et al.  Real-Time Tracking of Selective Auditory Attention From M/EEG: A Bayesian Filtering Approach , 2017, bioRxiv.

[16]  John J. Foxe,et al.  Attentional Selection in a Cocktail Party Environment Can Be Decoded from Single-Trial EEG. , 2015, Cerebral cortex.

[17]  M. D'Zmura,et al.  Envelope responses in single-trial EEG indicate attended speaker in a ‘cocktail party’ , 2014, Journal of Neural Engineering.

[18]  René J. Huster,et al.  EEG-Neurofeedback as a Tool to Modulate Cognition and Behavior: A Review Tutorial , 2017, Front. Hum. Neurosci..

[19]  Alexander Bertrand,et al.  Online detection of auditory attention with mobile EEG: closing the loop with neurofeedback , 2017, bioRxiv.

[20]  Antoine J. Shahin,et al.  Attentional Gain Control of Ongoing Cortical Speech Representations in a “Cocktail Party” , 2010, The Journal of Neuroscience.

[21]  J. Polich,et al.  Attention, probability, and task demands as determinants of P300 latency from auditory stimuli. , 1986, Electroencephalography and clinical neurophysiology.

[22]  Malcolm Slaney,et al.  A Comparison of Regularization Methods in Forward and Backward Models for Auditory Attention Decoding , 2018, Front. Neurosci..

[23]  David R Moore,et al.  Neural indices of listening effort in noisy environments , 2019, Scientific Reports.

[24]  F. Castellanos,et al.  Entrainment of neural oscillations as a modifiable substrate of attention , 2014, Trends in Cognitive Sciences.

[25]  Nima Mesgarani,et al.  Comparison of Two-Talker Attention Decoding from EEG with Nonlinear Neural Networks and Linear Methods , 2018 .

[26]  Mingjiang Wang,et al.  Identification of Auditory Object-Specific Attention from Single-Trial Electroencephalogram Signals via Entropy Measures and Machine Learning , 2018, Entropy.

[27]  Alexander Bertrand,et al.  The effect of head-related filtering and ear-specific decoding bias on auditory attention detection , 2016, Journal of neural engineering.

[28]  Adrian K. C. Lee,et al.  Auditory attention switching with listening difficulty: Behavioral and pupillometric measures. , 2018, Journal of the Acoustical Society of America.

[29]  Alexander Bertrand,et al.  EEG-based auditory attention detection: boundary conditions for background noise and speaker positions. , 2018, Journal of neural engineering.

[30]  Zhuo Chen,et al.  Neural decoding of attentional selection in multi-speaker environments without access to clean sources , 2017, Journal of neural engineering.

[31]  Daniel J. Strauss,et al.  Electrophysiological correlates of listening effort: neurodynamical modeling and measurement , 2010, Cognitive Neurodynamics.

[32]  Tom Francart,et al.  Top-down modulation of neural envelope tracking: the interplay with behavioral, self-report and neural measures of listening effort , 2019, bioRxiv.

[33]  Mikko Sams,et al.  Selective Attention Increases Both Gain and Feature Selectivity of the Human Auditory Cortex , 2007, PloS one.

[34]  F. Bloom,et al.  Modulation of early sensory processing in human auditory cortex during auditory selective attention. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Murat Akçakaya,et al.  EEG-assisted modulation of sound sources in the auditory scene , 2016, Biomed. Signal Process. Control..

[36]  Masoud Geravanchizadeh,et al.  Selective auditory attention detection based on effective connectivity by single-trial EEG , 2020, Journal of neural engineering.

[37]  Victor J. Boucher,et al.  The Role of Low-frequency Neural Oscillations in Speech Processing: Revisiting Delta Entrainment , 2019, Journal of Cognitive Neuroscience.

[38]  Thomas Lunner,et al.  A Tutorial on Auditory Attention Identification Methods , 2019, Front. Neurosci..

[39]  J. Peelle.  Listening Effort: How the Cognitive Consequences of Acoustic Challenge Are Reflected in Brain and Behavior , 2017, Ear and hearing.