Using spontaneous eye blink-related brain activity to investigate cognitive load during mobile map-assisted navigation

The continuous assessment of pedestrians’ cognitive load during a naturalistic mobile map-assisted navigation task is challenging because of limited experimental control over stimulus presentation, human-map-interactions, and other participant responses. To overcome this challenge, the present study takes advantage of navigators’ spontaneous eye blinks during navigation to serve as event markers in continuously recorded electroencephalography (EEG) data to assess cognitive load in a mobile map-assisted navigation task. We examined if and how displaying different numbers of landmarks (3 vs. 5 vs. 7) on mobile maps along a given route would influence navigators’ cognitive load during navigation in virtual urban environments. Cognitive load was assessed by the peak amplitudes of the blink-related fronto-central N2 and parieto-occipital P3. Our results show increased parieto-occipital P3 amplitude indicating higher cognitive load in the 7-landmark condition, compared to showing 3 or 5 landmarks. Our prior research already demonstrated that participants acquire more spatial knowledge in the 5- and 7-landmark conditions compared to the 3-landmark condition. Together with the current study, we find that showing 5 landmarks, compared to 3 or 7 landmarks, improved spatial learning without overtaxing cognitive load during navigation in different urban environments. Our findings also indicate a possible cognitive load spillover effect during map-assisted wayfinding whereby cognitive load during map viewing might have affected cognitive load during goal-directed locomotion in the environment or vice versa. Our research demonstrates that users’ cognitive load and spatial learning should be considered together when designing the display of future navigation aids and that navigators’ eye blinks can serve as useful event makers to parse continuous human brain dynamics reflecting cognitive load in naturalistic settings.

[1]  K. Gramann,et al.  The effect of landmark visualization in mobile maps on brain activity during navigation: A virtual reality study , 2022, Frontiers in Virtual Reality.

[2]  K. Gramann,et al.  The BeMoBIL Pipeline for automated analyses of multimodal mobile brain and body imaging data , 2022, bioRxiv.

[3]  Ashutosh Kumar Singh,et al.  Redesigning navigational aids using virtual global landmarks to improve spatial knowledge retrieval , 2022, npj Science of Learning.

[4]  E. Wascher,et al.  Visual Demands of Walking Are Reflected in Eye-Blink-Evoked EEG-Activity , 2022, Applied Sciences.

[5]  M. Kimura,et al.  Investigation of the optimal time interval between task-irrelevant auditory probes for evaluating mental workload in the shortest possible time. , 2022, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[6]  Toru Ishikawa,et al.  Online mobile map effect: how smartphone map use impairs spatial memory , 2021, Spatial Cogn. Comput..

[7]  K. Gramann,et al.  Eye Blink-Related Brain Potentials During Landmark-Based Navigation in Virtual Reality (Short Paper) , 2022, COSIT.

[8]  Jesus G. Cruz-Garza,et al.  Evaluating the impacts of color, graphics, and architectural features on wayfinding in healthcare settings using EEG data and virtual response testing , 2021, Journal of Environmental Psychology.

[9]  Niels A. Kloosterman,et al.  Zapline‐plus: A Zapline extension for automatic and adaptive removal of frequency‐specific noise artifacts in M/EEG , 2021, bioRxiv.

[10]  Aurélie Dommes,et al.  How does the visual and cognitive saliency of landmarks improve construction of spatial representations in younger and older adults? , 2021, Spatial Cogn. Comput..

[11]  Sara Irina Fabrikant,et al.  The influence of landmark visualization style on expert wayfinders' visual attention during a real-world navigation task , 2021 .

[12]  Anthony J. Ries,et al.  Gaze Behavior During Navigation and Visual Search of an Open-World Virtual Environment , 2021, Frontiers in Psychology.

[13]  Tien-Thong Nguyen Do,et al.  Human brain dynamics in active spatial navigation , 2021, Scientific Reports.

[14]  Jesus G. Cruz-Garza,et al.  Comparing physiological responses during cognitive tests in virtual environments vs. in identical real-world environments , 2021, Scientific Reports.

[15]  D. Yesiltepe,et al.  Landmarks in wayfinding: a review of the existing literature , 2021, Cognitive Processing.

[16]  Mar Gonzalez-Franco,et al.  Rethinking GPS navigation: creating cognitive maps through auditory clues , 2020, Scientific Reports.

[17]  Johannes Schöning,et al.  MapRecorder: analysing real-world usage of mobile map applications , 2020, Behav. Inf. Technol..

[18]  K. Gramann,et al.  Human cortical dynamics during full-body heading changes , 2018, Scientific Reports.

[19]  Thora Tenbrink,et al.  Point Me In the Right Direction: Understanding User Behaviour with As-The-Crow-Flies Navigation , 2020, MobileHCI.

[20]  Klaus Gramann,et al.  Identifying key factors for improving ICA-based decomposition of EEG data in mobile and stationary experiments , 2020, bioRxiv.

[21]  Otmar Bock,et al.  Acquisition of landmark, route, and survey knowledge in a wayfinding task: in stages or in parallel? , 2020, Psychological Research.

[22]  Denis Michel,et al.  Assisting Pedestrian Wayfinding in urban Settings , 2020 .

[23]  Klaus Gramann,et al.  Eye-movement related brain potentials during assisted navigation in real-world , 2020, bioRxiv.

[24]  V. Bohbot,et al.  Habitual use of GPS negatively impacts spatial memory during self-guided navigation , 2020, Scientific Reports.

[25]  Zoe N. Gastelum,et al.  Where are we going and where have we been? Examining the effects of maps on spatial learning in an indoor guided navigation task , 2020, Cognitive Research: Principles and Implications.

[26]  Zoe N. Gastelum,et al.  Where are we going and where have we been? Examining the effects of maps on spatial learning in an indoor guided navigation task , 2020, Cognitive Research: Principles and Implications.

[27]  Frank Dickmann,et al.  Effects of visual map complexity on the attentional processing of landmarks , 2020, PloS one.

[28]  Tal Makovski,et al.  A Metacognitive Perspective of Visual Working Memory With Rich Complex Objects , 2020, Frontiers in Psychology.

[29]  Sara Irina Fabrikant,et al.  The advantage of globally visible landmarks for spatial learning , 2020 .

[30]  Anna Shafer-Skelton,et al.  Scaling up visual attention and visual working memory to the real world , 2020, Psychology of Learning and Motivation.

[31]  Julia Frankenstein,et al.  Wayfinding and acquisition of spatial knowledge with navigation assistance. , 2020, Journal of experimental psychology. Applied.

[32]  Rui Li,et al.  Spatial Learning in Smart Applications: Enhancing Spatial Awareness through Visualized Off-Screen Landmarks on Mobile Devices , 2019, Smart Spaces and Places.

[33]  K. Gramann,et al.  Landmark-Based Navigation Instructions Improve Incidental Spatial Knowledge Acquisition in Real-World Environments , 2019 .

[34]  Pascale Piolino,et al.  Virtual reality: A new method to investigate cognitive load during navigation , 2019, Journal of Environmental Psychology.

[35]  Kai-Florian Richter,et al.  The Future of Geographic Information Displays from GIScience, Cartographic, and Cognitive Science Perspectives (Vision Paper) , 2019, COSIT.

[36]  Ian T. Ruginski,et al.  GPS use negatively affects environmental learning through spatial transformation abilities , 2019, Journal of Environmental Psychology.

[37]  P. Kiefer,et al.  FeaturEyeTrack: automatic matching of eye tracking data with map features on interactive maps , 2019, GeoInformatica.

[38]  Kai-Florian Richter,et al.  How does navigation system behavior influence human behavior? , 2019, Cognitive research: principles and implications.

[39]  Kenneth Kreutz-Delgado,et al.  ICLabel: An automated electroencephalographic independent component classifier, dataset, and website , 2019, NeuroImage.

[40]  Stevens,et al.  Is Simpler Always Better? Effects of Perceptual Detail and Viewpoint on Spatial Cognition and Metacognition , 2019, The American Journal of Psychology.

[41]  S. Page,et al.  Directional behavioral spillover and cognitive load effects in multiple repeated games , 2019 .

[42]  Toru Ishikawa,et al.  Satellite Navigation and Geospatial Awareness: Long-Term Effects of Using Navigation Tools on Wayfinding and Spatial Orientation , 2018, The Professional Geographer.

[43]  Arne D. Ekstrom,et al.  Which way is the bookstore? A closer look at the judgments of relative directions task , 2018, bioRxiv.

[44]  Jakub Hort,et al.  Spatial navigation deficits — overlooked cognitive marker for preclinical Alzheimer disease? , 2018, Nature Reviews Neurology.

[45]  Olaf Dimigen,et al.  Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis , 2018, bioRxiv.

[46]  Robert E. Roth,et al.  Mobile Maps and Responsive Design , 2018 .

[47]  Weihua Dong,et al.  Exploring differences of visual attention in pedestrian navigation when using 2D maps and 3D geo-browsers , 2017 .

[48]  Johannes Schöning,et al.  Pharos: improving navigation instructions on smartwatches by including global landmarks , 2017, MobileHCI.

[49]  T. Schubert,et al.  Comparison of the Working Memory Load in N-Back and Working Memory Span Tasks by Means of EEG Frequency Band Power and P300 Amplitude , 2017, Front. Hum. Neurosci..

[50]  Timothy F. Brady,et al.  Working memory is not fixed-capacity: More active storage capacity for real-world objects than for simple stimuli , 2016, Proceedings of the National Academy of Sciences.

[51]  E. Wascher,et al.  Age-Sensitive Effects of Enduring Work with Alternating Cognitive and Physical Load. A Study Applying Mobile EEG in a Real Life Working Scenario , 2016, Front. Hum. Neurosci..

[52]  Colin G. Ellard,et al.  Cognitive and affective responses to natural scenes: Effects of low level visual properties on preference, cognitive load and eye-movements , 2015 .

[53]  Tad T. Brunyé,et al.  Navigational Aids and Spatial Memory Impairment: The Role of Divided Attention , 2015, Spatial Cogn. Comput..

[54]  Antoni Moore,et al.  ‘Web Cartography – Map Design for Interactive and Mobile Devices’ , 2015 .

[55]  Aiden E. G. F. Arnold,et al.  A critical review of the allocentric spatial representation and its neural underpinnings: toward a network-based perspective , 2014, Front. Hum. Neurosci..

[56]  Motohiro Kimura,et al.  Electrophysiological measurement of interest during walking in a simulated environment. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[57]  A. Ekstrom,et al.  Different “routes” to a cognitive map: dissociable forms of spatial knowledge derived from route and cartographic map learning , 2014, Memory & cognition.

[58]  Mary C Potter,et al.  Large capacity temporary visual memory. , 2014, Journal of experimental psychology. General.

[59]  Russell A. Epstein,et al.  Neural systems for landmark-based wayfinding in humans , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[60]  Stephan Winter,et al.  Introduction: What Landmarks Are, and Why They Are Important , 2014 .

[61]  Sven Hoffmann,et al.  Towards the measurement of event-related EEG activity in real-life working environments. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[62]  Tad T. Brunyé,et al.  How Navigational Aids Impair Spatial Memory: Evidence for Divided Attention , 2013, Spatial Cogn. Comput..

[63]  Elizabeth R. Chrastil.  Neural evidence supports a novel framework for spatial navigation , 2013, Psychonomic bulletin & review.

[64]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[65]  Elizabeth R. Chrastil,et al.  Active and passive spatial learning in human navigation: acquisition of survey knowledge. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[66]  H. Zimmer,et al.  Navigation assistance: a trade-off between wayfinding support and configural learning support. , 2012, Journal of experimental psychology. Applied.

[67]  G. Gartner,et al.  Spatial Knowledge Acquisition with Mobile Maps, Augmented Reality and Voice in the Context of GPS-based Pedestrian Navigation: Results from a Field Test , 2012 .

[68]  Steven J. Luck,et al.  Event-related potentials. , 2012 .

[69]  David Rindskopf,et al.  APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychometrics. , 2012 .

[70]  D. Uttal,et al.  Spatial Thinking and STEM Education. When, Why, and How? , 2012 .

[71]  Tracy Xiao Liu,et al.  Behavioral spillovers and cognitive load in multiple games: An experimental study , 2012, Games Econ. Behav..

[72]  Kenneth Kreutz-Delgado,et al.  AMICA : An Adaptive Mixture of Independent Component Analyzers with Shared Components , 2011 .

[73]  Stephan Winter,et al.  Including landmarks in routing instructions , 2010, J. Locat. Based Serv..

[74]  S. Steinhauer,et al.  Blink before and after you think: blinks occur prior to and following cognitive load indexed by pupillary responses. , 2008, Psychophysiology.

[75]  G. Fitzmaurice,et al.  Advances in longitudinal data analysis: An historical perspective , 2008 .

[76]  Jonathan R. Folstein,et al.  Influence of cognitive control and mismatch on the N2 component of the ERP: a review. , 2007, Psychophysiology.

[77]  Avi Parush,et al.  Degradation in Spatial Knowledge Acquisition When Using Automatic Navigation Systems , 2007, COSIT.

[78]  Andrew Gelman,et al.  Multilevel (Hierarchical) Modeling: What It Can and Cannot Do , 2006, Technometrics.

[79]  Donald Hedeker,et al.  Longitudinal Data Analysis , 2006 .

[80]  A. Miyake,et al.  The Cambridge Handbook of Visuospatial Thinking , 2005 .

[81]  D. R. Montello.  The Cambridge Handbook of Visuospatial Thinking: Navigation , 2005 .

[82]  John W. Philbeck,et al.  Remembered landmarks enhance the precision of path integration. , 2005 .

[83]  R. T. Pivik,et al.  Endogenous eye blinks in preadolescents: relationship to information processing and performance , 2004, Biological Psychology.

[84]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[85]  A. Baddeley.  Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[86]  R. Oostenveld,et al.  Validating the boundary element method for forward and inverse EEG computations in the presence of a hole in the skull , 2002, Human brain mapping.

[87]  Stephan Winter,et al.  Enriching Wayfinding Instructions with Local Landmarks , 2002, GIScience.

[88]  Daniel R. Montello,et al.  Cognitive Map-Design Research in the Twentieth Century: Theoretical and Empirical Approaches , 2002 .

[89]  A. Kok.  On the utility of P3 amplitude as a measure of processing capacity. , 2001, Psychophysiology.

[90]  G. Woodman,et al.  Storage of features, conjunctions and objects in visual working memory. , 2001, Journal of experimental psychology. Human perception and performance.

[91]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[92]  Hideki Ohira,et al.  Effects of Stimulus Valence on Recognition Memory and Endogenous Eyeblinks: Further Evidence for Positive-Negative Asymmetry , 1998 .

[93]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[94]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[95]  Risto N t nen.  Attention and brain function , 1992 .

[96]  J. Stern,et al.  Blinks as an index of cognitive activity during reading , 1991, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[97]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[98]  P Berg,et al.  Eyeblink-related potentials. , 1988, Electroencephalography and clinical neurophysiology.

[99]  J. Stern,et al.  The endogenous eyeblink. , 1984, Psychophysiology.

[100]  A. Siegel,et al.  The development of spatial representations of large-scale environments. , 1975, Advances in child development and behavior.