Towards the assessment of listening effort in real life situations: Mobile EEG recordings in a multimodal driving situation

The EEG estimation of listening effort has been proven to efficiently map subjectively perceived effort on an objective scale. However, it has mostly been performed in controlled audiometric laboratory environments. In this study, we investigated the feasibility of the EEG listening effort assessment in a multisensory demanding environment. For this purpose, two mobile EEG acquisition systems were used to extract the objective listening effort of the ongoing oscillatory activity (OLEosc) while driving. As an auditory paradigm was presented to trigger listening effort, the simulated driving scenario served as a competing task responding to different modalities. The OLEosc was compared with a subjective rating of the expended listening effort and the task performance. Furthermore, we considered time resolved listening effort profiles over the duration of the paradigm. The results showed that the practical assessment of listening effort in a multimodal real life setting is viable. The OLEosc could be extracted and followed the subjective listening effort. Moreover, the analysis of the dynamic listening effort profiles unveiled further information such as “surrender effects”, when the subjects ceased to solve the auditory task due to the intense multimodal load.

[1]  C. Spence,et al.  Cross-modal links in spatial attention. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[2]  Daniel J. Strauss,et al.  Electrophysiological correlates of listening effort: neurodynamical modeling and measurement , 2010, Cognitive Neurodynamics.

[3]  Philipp Berens,et al.  CircStat: AMATLABToolbox for Circular Statistics , 2009, Journal of Statistical Software.

[4]  J. Driver,et al.  Crossmodal links in endogenous and exogenous spatial attention: evidence from event-related brain potential studies , 2001, Neuroscience & Biobehavioral Reviews.

[5]  Daniel J. Strauss,et al.  Objective assessment of listening effort in the oscillatory EEG: Comparison of different hearing aid configurations , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  Jean-Pierre Gagné,et al.  Use of a Dual-Task Paradigm to Measure Listening Effort Utilisation d ’ un paradigme de double tâche pour mesurer l ’ attention auditive , 2010 .

[7]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[8]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[9]  J. Driver,et al.  Audiovisual links in endogenous covert spatial attention. , 1996, Journal of experimental psychology. Human perception and performance.

[10]  Marty G. Woldorff,et al.  Selective Attention and Multisensory Integration: Multiple Phases of Effects on the Evoked Brain Activity , 2005, Journal of Cognitive Neuroscience.

[11]  B. Stein,et al.  The Merging of the Senses , 1993 .

[12]  P. Bertelson Chapter 14 Ventriloquism: A case of crossmodal perceptual grouping , 1999 .

[13]  Daniel J. Strauss,et al.  Neural correlates of listening effort related factors: Influence of age and hearing impairment , 2013, Brain Research Bulletin.

[14]  Jon Driver,et al.  Cross-Modal Interactions between Audition, Touch, and Vision in Endogenous Spatial Attention: ERP Evidence on Preparatory States and Sensory Modulations , 2002, Journal of Cognitive Neuroscience.

[15]  Martin Eimer Electrophysiolgy of Human Crossmodal Spatial Attention , 2004 .