EEG-based cognitive load of processing events in 3D virtual worlds is lower than processing events in 2D displays.

Interacting with 2D displays, such as computer screens, smartphones, and TV, is currently a part of our daily routine; however, our visual system is built for processing 3D worlds. We examined the cognitive load associated with a simple and a complex task of learning paper-folding (origami) by observing 2D or stereoscopic 3D displays. While connected to an electroencephalogram (EEG) system, participants watched a 2D video of an instructor demonstrating the paper-folding tasks, followed by a stereoscopic 3D projection of the same instructor (a digital avatar) illustrating identical tasks. We recorded the power of alpha and theta oscillations and calculated the cognitive load index (CLI) as the ratio of the average power of frontal theta (Fz.) and parietal alpha (Pz). The results showed a significantly higher cognitive load index associated with processing the 2D projection as compared to the 3D projection; additionally, changes in the average theta Fz power were larger for the 2D conditions as compared to the 3D conditions, while alpha average Pz power values were similar for 2D and 3D conditions for the less complex task and higher in the 3D state for the more complex task. The cognitive load index was lower for the easier task and higher for the more complex task in 2D and 3D. In addition, participants with lower spatial abilities benefited more from the 3D compared to the 2D display. These findings have implications for understanding cognitive processing associated with 2D and 3D worlds and for employing stereoscopic 3D technology over 2D displays in designing emerging virtual and augmented reality applications.

[1]  Mark Peterson,et al.  Towards a Research Agenda for the Use of Three-Dimensional Virtual Worlds in Language Learning , 2011 .

[2]  Matthew W. Miller,et al.  A novel approach to the physiological measurement of mental workload. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  Eunjoon Rachel Um,et al.  Emotional design in multimedia learning. , 2012 .

[4]  Min-Chul Park,et al.  Overview of Measurement Methods for Factors Affecting the Human Visual System in 3D Displays , 2015, Journal of Display Technology.

[5]  J. Kropotov.  Quantitative EEG, Event-Related Potentials and Neurotherapy , 2008 .

[6]  Miriam Reiner,et al.  Multimodal Virtual Environments: Response Times, Attention, and Presence , 2006, PRESENCE: Teleoperators and Virtual Environments.

[7]  Jyh-Chong Liang,et al.  Current status, opportunities and challenges of augmented reality in education , 2013, Comput. Educ..

[8]  D. Leutner,et al.  Direct Measurement of Cognitive Load in Multimedia Learning , 2003 .

[9]  J. Sweller.  COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[10]  Tzyy-Ping Jung,et al.  Brain oscillation and connectivity during a chemistry visual working memory task. , 2013, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[11]  D. Sujan,et al.  Origami Theory and Its Applications: A Literature Review , 2013 .

[12]  R. Kass,et al.  Automatic correction of ocular artifacts in the EEG: a comparison of regression-based and component-based methods , 2004 .

[13]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[14]  Nian-Shing Chen,et al.  Taking a signal: A review of gesture-based computing research in education , 2014, Comput. Educ..

[15]  A. Takeuchi,et al.  Brain lattice: Concept lattice based causal analysis of changes in mental workload , 2013, 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA).

[16]  Satu Palva,et al.  Load Dependence of β and γ Oscillations Predicts Individual Capacity of Visual Attention , 2013, The Journal of Neuroscience.

[17]  Richard Catrambone,et al.  Designing Instructional Examples to Reduce Intrinsic Cognitive Load: Molar versus Modular Presentation of Solution Procedures , 2004 .

[18]  Igor Crk,et al.  Toward using alpha and theta brain waves to quantify programmer expertise , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  Takeo Kanade,et al.  A Theory of Origami World , 1979, Artif. Intell..

[20]  Monica Z. Weiland,et al.  Real Time Research Methods , 2013 .

[21]  Barney Dalgarno,et al.  What are the learning affordances of 3-D virtual environments? , 2010, Br. J. Educ. Technol..

[22]  Pavlo D. Antonenko,et al.  Using Electroencephalography to Measure Cognitive Load , 2010 .

[23]  Marilyn Tremaine,et al.  Understanding visualization through spatial ability differences , 2005, VIS 05. IEEE Visualization, 2005..

[24]  A. Testa,et al.  On the Processing of harmonics and interharmonics: using Hanning window in standard framework , 2004, IEEE Transactions on Power Delivery.

[25]  Brendan Z. Allison,et al.  Workload assessment of computer gaming using a single-stimulus event-related potential paradigm , 2008, Biological Psychology.

[26]  Richard E. Mayer,et al.  A Cognitive Theory of Multimedia Learning: Implications for Design Principles , 2001 .

[27]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[28]  Yvonne Kammerer,et al.  Pupil Dilation and EEG Alpha Frequency Band Power Reveal Load on Executive Functions for Link-Selection Processes during Text Reading , 2015, PloS one.

[29]  C. Mélan,et al.  What is the relationship between mental workload factors and cognitive load types? , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  Ruth B. Ekstrom.  Cognitive factors: Their identification and replication. , 1979 .

[31]  W J Levy,et al.  Effect of Epoch Length on Power Spectrum Analysis of the EEG , 1987, Anesthesiology.

[32]  Michael Vourkas,et al.  Changes in Linear and Nonlinear EEG Measures as a Function of Task Complexity: Evidence for Local and Distant Signal Synchronization , 2004, Brain Topography.

[33]  I. Gere,et al.  Multimedia: Differences in cognitive processes observed with EEG , 1999 .

[34]  Antonio Frisoli,et al.  Understanding and Realizing Presence in the Presenccia Project , 2007, IEEE Computer Graphics and Applications.

[35]  Bart van den Hooff,et al.  Lost in space? Cognitive fit and cognitive load in 3D virtual environments , 2013, Comput. Hum. Behav..

[36]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[37]  Norbert Jaušovec,et al.  Working memory training: Improving intelligence – Changing brain activity , 2012, Brain and Cognition.

[38]  E. Basar,et al.  Gamma, alpha, delta, and theta oscillations govern cognitive processes. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[39]  Desney S. Tan,et al.  Feasibility and pragmatics of classifying working memory load with an electroencephalograph , 2008, CHI.

[40]  J. Sweller.  Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load , 2010 .

[41]  T Janzen,et al.  Differences in baseline EEG measures for ADD and Normally Achieving preadolescent males , 1995, Biofeedback and self-regulation.

[42]  R. Mayer,et al.  Cognitive Principles of Multimedia Learning: The Role of Modality and Contiguity , 1999 .

[43]  Paul Chandler,et al.  Contemporary cognitive load theory research: The good, the bad and the ugly , 2011, Comput. Hum. Behav..

[44]  Mehmet Göktürk,et al.  Psychophysiological measures of human cognitive states applied in human computer interaction , 2011, WCIT.

[45]  Maarten A. Hogervorst,et al.  Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload , 2014, Front. Neurosci..

[46]  Slava Kalyuga,et al.  Instructional designs for the development of transferable knowledge and skills: A cognitive load perspective , 2009, Comput. Hum. Behav..

[47]  U. Goswami.  Mind, Brain, and Literacy: Biomarkers as Usable Knowledge for Education , 2009 .

[48]  Kristian Lukander,et al.  Estimating Brain Load from the EEG , 2009, TheScientificWorldJournal.

[49]  T. Gog,et al.  Development of an instrument for measuring different types of cognitive load , 2013, Behavior Research Methods.

[50]  Martha E. Crosby,et al.  Assessing Cognitive Load with Physiological Sensors , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[51]  W. Klimesch.  EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[52]  Nigel H. Lovell,et al.  Estimating cognitive workload using wavelet entropy-based features during an arithmetic task , 2013, Comput. Biol. Medicine.

[53]  Miriam Reiner,et al.  Estimating mental workload through event-related fluctuations of pupil area during a task in a virtual world. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[54]  C. Wickens,et al.  Situation Awareness, Mental Workload, and Trust in Automation: Viable, Empirically Supported Cognitive Engineering Constructs , 2008 .

[55]  Yasin Ozarslan,et al.  Augmented Reality in Education: Current Technologies and the Potential for Education , 2012 .

[56]  Wolfgang Rosenstiel,et al.  Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain-computer interface approach , 2014, Front. Neurosci..

[57]  F. Harris.  On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[58]  Laura Bonanni,et al.  Electrophysiological indices of interference resolution covary with individual fluid intelligence: investigating reactive control processes in a 3-back working memory task , 2014, NeuroImage.

[59]  C. Neuper,et al.  EEG alpha band dissociation with increasing task demands. , 2005, Brain research. Cognitive brain research.

[60]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[61]  Daniel Brandeis,et al.  Frontal Midline Theta Reflects Individual Task Performance in a Working Memory Task , 2014, Brain Topography.

[62]  Carolyne R Swain,et al.  EEG power changes during a multiple level memory retention task. , 1999, International Journal of Psychophysiology.

[63]  Tassos A. Mikropoulos,et al.  Educational virtual environments: A ten-year review of empirical research (1999-2009) , 2011, Comput. Educ..

[64]  William A. Crossland,et al.  53.1: Real-time Binary Hologram Generation for High-quality Video Projection Applications , 2004 .

[65]  F. Rösler,et al.  Theta and alpha oscillations during working-memory maintenance predict successful long-term memory encoding , 2010, Neuroscience Letters.

[66]  Liesbeth Kester,et al.  Cognitive load theory and multimedia learning, task characteristics and learning engagement: The Current State of the Art , 2011, Comput. Hum. Behav..

[67]  E. Basar,et al.  Are cognitive processes manifested in event-related gamma, alpha, theta and delta oscillations in the EEG? , 1999, Neuroscience Letters.

[68]  J. Merriënboer,et al.  Research on cognitive load theory and its design implications for e-learning , 2005 .

[69]  Norbert Jausovec,et al.  Differences in induced gamma and upper alpha oscillations in the human brain related to verbal/performance and emotional intelligence. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[70]  K. Newell,et al.  Modulation of cortical activity in 2D versus 3D virtual reality environments: an EEG study. , 2015, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[71]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..