Classifying EEG Signals during Stereoscopic Visualization to Estimate Visual Comfort

With stereoscopic displays a sensation of depth that is too strong could impede visual comfort and may result in fatigue or pain. We used Electroencephalography (EEG) to develop a novel brain-computer interface that monitors users' states in order to reduce visual strain. We present the first system that discriminates comfortable conditions from uncomfortable ones during stereoscopic vision using EEG. In particular, we show that either changes in event-related potentials' (ERPs) amplitudes or changes in EEG oscillations power following stereoscopic objects presentation can be used to estimate visual comfort. Our system reacts within 1 s to depth variations, achieving 63% accuracy on average (up to 76%) and 74% on average when 7 consecutive variations are measured (up to 93%). Performances are stable (≈62.5%) when a simplified signal processing is used to simulate online analyses or when the number of EEG channels is lessened. This study could lead to adaptive systems that automatically suit stereoscopic displays to users and viewing conditions. For example, it could be possible to match the stereoscopic effect with users' state by modifying the overlap of left and right images according to the classifier output.

[1]  Sebastian Bosse,et al.  Toward a Direct Measure of Video Quality Perception Using EEG , 2012, IEEE Transactions on Image Processing.

[2]  Aaron W. Bangor,et al.  Display Technology and Ambient Illumination Influences on Visual Fatigue at VDT Workstations , 2000 .

[3]  Clemens Brunner,et al.  Better than random? A closer look on BCI results , 2008 .

[4]  Edward Cutrell,et al.  BCI for passive input in HCI , 2007 .

[5]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[6]  Jerome H. Friedman,et al.  On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality , 2004, Data Mining and Knowledge Discovery.

[7]  R. Ward,et al.  EMG and EOG artifacts in brain computer interface systems: A survey , 2007, Clinical Neurophysiology.

[8]  Hussein A. Abbass,et al.  Augmented Cognition using Real-time EEG-based Adaptive Strategies for Air Traffic Control , 2014 .

[9]  J F Thayer,et al.  Facial muscle activity and EEG recordings: redundancy analysis. , 1991, Electroencephalography and clinical neurophysiology.

[10]  Jérémy Frey,et al.  Continuous Mental Effort Evaluation During 3D Object Manipulation Tasks Based on Brain and Physiological Signals , 2015, INTERACT.

[11]  Sebastian Bosse,et al.  EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs) , 2015, Journal of neural engineering.

[12]  Alessandro Rizzi,et al.  Assessing stereo blindness and stereo acuity on digital displays , 2014, Displays.

[13]  Klaus-Robert Müller,et al.  Spatio-spectral filters for improving the classification of single trial EEG , 2005, IEEE Transactions on Biomedical Engineering.

[14]  Klaus-Robert Müller,et al.  Stereoscopic depth increases intersubject correlations of brain networks , 2014, NeuroImage.

[15]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[16]  J Farley Norman,et al.  Stereoscopic shape discrimination is well preserved across changes in object size. , 2009, Acta psychologica.

[17]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[18]  Constantine Stephanidis,et al.  3D Visualization and Multimodal Interaction with Temporal Information Using Timelines , 2013, INTERACT.

[19]  Bernd Fröhlich,et al.  Exploring geo-scientific data in virtual environments , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[20]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[21]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[22]  Atanas Gotchev,et al.  Relative importance of depth cues on portable autostereoscopic display , 2010, MoViD '10.

[23]  Martin Hachet,et al.  Toucheo: multitouch and stereo combined in a seamless workspace , 2011, UIST.

[24]  Weiqiang Dong.  On Bias , Variance , 0 / 1-Loss , and the Curse of Dimensionality RK April 13 , 2014 .

[25]  Jérémy Frey,et al.  Estimating Visual Comfort in Stereoscopic Displays Using Electroencephalography: A Proof-of-Concept , 2015, INTERACT.

[26]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[27]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.

[28]  Kun Li,et al.  EEG-based detection and evaluation of fatigue caused by watching 3DTV , 2013, Displays.

[29]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[30]  H. Momeni-Moghadam,et al.  Stereopsis with TNO and Titmus Tests in Symptomatic and Asymptomatic University Students , 2011 .

[31]  Min-Koo Kang,et al.  Feasibility study for visual discomfort assessment on stereo images using EEG , 2012, 2012 International Conference on 3D Imaging (IC3D).

[32]  Chi Thanh Vi,et al.  Detecting error-related negativity for interaction design , 2012, CHI.

[33]  G. Pfurtscheller,et al.  A fully automated correction method of EOG artifacts in EEG recordings , 2007, Clinical Neurophysiology.

[34]  Keetaek Kham,et al.  Measurement of 3D Visual Fatigue Using Event-Related Potential (ERP): 3D Oddball Paradigm , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[35]  Stefan Haufe,et al.  The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology , 2010, Front. Neurosci..

[36]  Christian Mühl,et al.  Review of the Use of Electroencephalography as an Evaluation Method for Human-Computer Interaction , 2013, PhyCS.

[37]  Elsa Andrea Kirchner,et al.  Effects of eye artifact removal methods on single trial P300 detection, a comparative study , 2014, Journal of Neuroscience Methods.

[38]  Kang Ryoung Park,et al.  Assessment of Eye Fatigue Caused by 3D Displays Based on Multimodal Measurements , 2014, Sensors.

[39]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[40]  A. Mognon,et al.  ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features. , 2011, Psychophysiology.

[41]  Pascal Mamassian,et al.  The Influence of Object Size and Surface Shape on Shape Constancy from Stereo , 2004, Perception.

[42]  Bernd Fröhlich,et al.  Collaborative Interaction in Co-Located Two-User Scenarios , 2009, EGVE/ICAT/EuroVR.

[43]  Neil A. Dodgson,et al.  Variation and extrema of human interpupillary distance , 2004, IS&T/SPIE Electronic Imaging.

[44]  Touradj Ebrahimi,et al.  Spatial filters for the classification of event-related potentials , 2006, ESANN.

[45]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .