A preliminary quantitative EEG study on Augmented Reality Guidance of Manual Tasks

Augmented reality (AR) head mounted displays (HMDs) combine user’s natural view of the real world with virtual data. Thus, they might be particularly suited to guide manual tasks as in computer-aided-surgery. However, the typical focal plane of commercial devices is outside the user’s peripersonal space (i.e. the space containing reachable objects), limiting the performance of manual-task guidance. Specifically, known issues such as the "vergence-accomodation-conflict" and the "focus-rivalry" may lead to visual fatigue and mental workload worsening task performance. Here, we exploit EEG recordings during a "connecting-the-dots" task performed with and without AR to study the effects of mental workload associated with AR-related visual fatigue. First, we quantify the reduction of users’ performance based on starting and end points gap errors. Then, we investigate the effects on AR usage on cortical activity through the analysis of EEG power and Frontal Alpha Asymmetry (FAA) index. Although preliminary, EEG power results suggest that mental workload associated with AR usage may derive from enhanced difficulty associated with the task.

[1]  Yongtian Wang,et al.  EEG-Based Assessment of Stereoscopic 3D Visual Fatigue Caused by Vergence-Accommodation Conflict , 2015, Journal of Display Technology.

[2]  Jérémy Frey,et al.  Assessing the zone of comfort in stereoscopic displays using EEG , 2014, CHI Extended Abstracts.

[3]  Mariam Hassib,et al.  Estimating Visual Discomfort in Head-Mounted Displays Using Electroencephalography , 2017, INTERACT.

[4]  P. Ekman,et al.  Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology. I. , 1990, Journal of personality and social psychology.

[5]  Sara Condino,et al.  Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks , 2020, IEEE Transactions on Biomedical Engineering.

[6]  Stephan P. Swinnen,et al.  Ipsilateral coordination at preferred rate: Effects of age, body side and task complexity , 2009, NeuroImage.

[7]  Wolfgang Birkfellner,et al.  Development of the Varioscope AR. A see-through HMD for computer-aided surgery , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[8]  Gregory Kramida,et al.  Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays , 2016, IEEE Transactions on Visualization and Computer Graphics.

[9]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.

[10]  Keetaek Kham,et al.  Measurement of 3D Visual Fatigue Using Event-Related Potential (ERP): 3D Oddball Paradigm , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[11]  Milan Sonka,et al.  Computer-aided liver surgery planning: an augmented reality approach , 2003, SPIE Medical Imaging.

[12]  Kenneth Kreutz-Delgado,et al.  AMICA : An Adaptive Mixture of Independent Component Analyzers with Shared Components , 2011 .

[13]  Tzyy-Ping Jung,et al.  Real-time neuroimaging and cognitive monitoring using wearable dry EEG , 2015, IEEE Transactions on Biomedical Engineering.

[14]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[15]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[16]  Jeannine Herron,et al.  Effects of task difficulty on EEG measures of cerebral engagement , 1978, Neuropsychologia.

[17]  Meng Wang,et al.  A Wearable SSVEP-Based BCI System for Quadcopter Control Using Head-Mounted Device , 2018, IEEE Access.

[18]  David Budgen,et al.  Evaluating Subjective Impressions of Quality Controlled 3D Films on Large and Small Screens , 2015, Journal of Display Technology.

[19]  E. Peli,et al.  The visual effects of head-mounted display (HMD) are not distinguishable from those of desk-top computer display , 1998, Vision Research.

[20]  R. Ivry,et al.  Ipsilateral motor cortex activity during unimanual hand movements relates to task complexity. , 2005, Journal of neurophysiology.

[21]  Supun Samarasekera,et al.  AR-Weapon: Live Augmented Reality Based First-Person Shooting System , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[22]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[23]  Benny B. Briesemeister,et al.  Approach the Good, Withdraw from the Bad—A Review on Frontal Alpha Asymmetry Measures in Applied Psychological Research , 2013 .

[24]  John J. B. Allen,et al.  Assessing and conceptualizing frontal EEG asymmetry: An updated primer on recording, processing, analyzing, and interpreting frontal alpha asymmetry. , 2017, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[25]  Hirokazu Kato,et al.  SharpView: Improved clarity of defocussed content on optical see-through head-mounted displays , 2016, 2016 IEEE Virtual Reality (VR).