Changes of EEG phase synchronization and EOG signals along the use of steady state visually evoked potential-based brain computer interface

The steady-state visual evoked potential (SSVEP)-based brain computer interface (BCI) has demonstrated relatively high performance with little user training, and thus becomes a popular BCI paradigm. However, due to the performance deterioration over time, its robustness and reliability appear not sufficient to allow a non-expert to use outside laboratory. It would be thus helpful to study what happens behind the decreasing tendency of the BCI performance. This paper explores the changes of brain networks and electrooculography (EOG) signals to investigate the cognitive capability changes along the use of the SSVEP-based BCI. The EOG signals are characterized by the blink amplitudes and the speeds of saccades, and the brain networks are estimated by the instantaneous phase synchronizations of electroencephalography (EEG) signals. Experimental results revealed that the characteristics derived from EOG and brain networks have similar trends which contain two stages. At the beginning, the blink amplitudes and the saccade speeds start to reduce. Meanwhile, the global synchronizations of the brain networks are formed quickly. These observations implies that the cognitive decline along the use of the SSVEP-based BCI. Then, the EOG and the brain networks related characteristics demonstrate a slow recovery or relatively stable trend. This study could be helpful for a better understanding about the depreciation of the BCI performance as well as its relationship with the brain networks and the EOG along the use of the SSVEP-based BCI.

[1]  C. Stam,et al.  EEG Resting State Functional Connectivity in Adult Dyslexics Using Phase Lag Index and Graph Analysis , 2018, Front. Hum. Neurosci..

[2]  W. R. Miles,et al.  Eye movement and visual fixation during profound sleepiness. , 1931 .

[3]  Ramesh Srinivasan,et al.  Functional connectivity of frontal cortex in healthy and ADHD children reflected in EEG coherence. , 2007, Cerebral cortex.

[4]  J. Palva,et al.  New vistas for α-frequency band oscillations , 2007, Trends in Neurosciences.

[5]  J. C. Miller,et al.  Electrooculographic and performance indices of fatigue during simulated flight , 1996, Biological Psychology.

[6]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.

[7]  D. LaBerge Attention, Awareness, and the Triangular Circuit , 1997, Consciousness and Cognition.

[8]  R J Leigh,et al.  Properties of horizontal saccades accompanied by blinks. , 1998, Journal of neurophysiology.

[9]  Guanrong Chen,et al.  An adaptive optimal-Kernel time-frequency representation-based complex network method for characterizing fatigued behavior using the SSVEP-based BCI system , 2018, Knowl. Based Syst..

[10]  Xiaofang Yuan,et al.  A Regression Method With Subnetwork Neurons for Vigilance Estimation Using EOG and EEG , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[11]  Gareth J. Barker,et al.  Functional connectivity changes associated with fMRI neurofeedback of right inferior frontal cortex in adolescents with ADHD , 2019, NeuroImage.

[12]  Yijun Wang,et al.  Brain-Computer Interfaces Based on Visual Evoked Potentials , 2008, IEEE Engineering in Medicine and Biology Magazine.

[13]  Mohammad Pooyan,et al.  SSVEP recognition by modeling brain activity using system identification based on Box-Jenkins model , 2018, Comput. Biol. Medicine.

[14]  Suzanne Wasylink,et al.  Resting state functional connectivity predicts neurofeedback response , 2014, Front. Behav. Neurosci..

[15]  L. Colzato,et al.  Spontaneous eye blink rate as predictor of dopamine-related cognitive function—A review , 2016, Neuroscience & Biobehavioral Reviews.

[16]  Devin K. Binder,et al.  A resting EEG study of neocortical hyperexcitability and altered functional connectivity in fragile X syndrome , 2017, Journal of Neurodevelopmental Disorders.

[17]  Fabio Babiloni,et al.  Quantitative Assessment of the Training Improvement in a Motor-Cognitive Task by Using EEG, ECG and EOG Signals , 2015, Brain Topography.

[18]  H. Perrild,et al.  Undetectable thyrotropin levels using immunoradiometric assay in l-dopa-treated patients , 1988, Journal of Neurology.

[19]  R. Schleicher,et al.  Blinks and saccades as indicators of fatigue in sleepiness warners: looking tired? , 2022 .

[20]  Kathryn A. Lee,et al.  Validity and reliability of a scale to assess fatigue , 1991, Psychiatry Research.

[21]  Andreas Sprenger,et al.  Differential effects of sleep deprivation on saccadic eye movements. , 2005, Sleep.

[22]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[23]  C. Helmchen,et al.  Differential effects of blinks on horizontal saccade and smooth pursuit initiation in humans , 2004, Experimental Brain Research.

[24]  G. Kecklund,et al.  Sleepiness in long distance truck driving: an ambulatory EEG study of night driving. , 1993, Ergonomics.

[25]  Qianxiang Zhou,et al.  Electroencephalogram assessment of mental fatigue in visual search. , 2015, Bio-medical materials and engineering.

[26]  G Debus,et al.  Saccadic velocity and activation: development of a diagnostic tool for assessing energy regulation. , 1998, Ergonomics.

[27]  M. Finlayson,et al.  An evaluation of subjective and objective measures of fatigue in patients with brain injury and healthy controls. , 1998, Brain injury.

[28]  Dimitri Van De Ville,et al.  Dynamic reconfiguration of human brain functional networks through neurofeedback , 2013, NeuroImage.

[29]  B. Fischer,et al.  Visual field representations and locations of visual areas V1/2/3 in human visual cortex. , 2003, Journal of vision.

[30]  A. Craig,et al.  A critical review of the psychophysiology of driver fatigue , 2001, Biological Psychology.

[31]  Yu-Te Wang,et al.  A Comparison Study of Canonical Correlation Analysis Based Methods for Detecting Steady-State Visual Evoked Potentials , 2015, PloS one.

[32]  C. Evinger,et al.  Eyelid movements. Mechanisms and normal data. , 1991, Investigative ophthalmology & visual science.

[33]  W. Klimesch Memory processes, brain oscillations and EEG synchronization. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[34]  G. Sandini,et al.  Graph theoretical analysis of magnetoencephalographic functional connectivity in Alzheimer's disease. , 2009, Brain : a journal of neurology.

[35]  Juan Manuel Górriz,et al.  Brain Connectivity Analysis: A Short Survey , 2012, Comput. Intell. Neurosci..

[36]  H Eichenbaum,et al.  Thinking about brain cell assemblies. , 1993, Science.

[37]  Alpo Värri,et al.  Automatic detection of slow wave sleep using two channel electro-oculography , 2007, Journal of Neuroscience Methods.

[38]  V Latora,et al.  Efficient behavior of small-world networks. , 2001, Physical review letters.

[39]  Fabio Babiloni,et al.  Assessment of driving fatigue based on intra/inter-region phase synchronization , 2017, Neurocomputing.

[40]  W. Klimesch Alpha-band oscillations, attention, and controlled access to stored information , 2012, Trends in Cognitive Sciences.

[41]  Colin M. Shapiro,et al.  Visual Analogue Scale to Evaluate Fatigue Severity (VAS-F) , 2011 .

[42]  T. D. Papageorgiou,et al.  Brain–computer interfaces increase whole-brain signal to noise , 2013, Proceedings of the National Academy of Sciences.

[43]  Andreas Sprenger,et al.  Blink effect on slow vergence , 2002, Neuroreport.

[44]  Talib Rothengatter,et al.  Maintaining task set under fatigue: A study of time-on-task effects in simulated driving. , 2001 .

[45]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[46]  Stéphane Bonnet,et al.  Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[47]  Yijun Wang,et al.  A High-Speed SSVEP-Based BCI Using Dry EEG Electrodes , 2018, Scientific Reports.

[48]  Shangkai Gao,et al.  A practical VEP-based brain-computer interface. , 2006, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[49]  E W Banister,et al.  Modeling human performance in running. , 1990, Journal of applied physiology.

[50]  Martin Sarter,et al.  Augmented prefrontal acetylcholine release during challenged attentional performance. , 2006, Cerebral cortex.

[51]  Xiaogang Chen,et al.  Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain–computer interface , 2015, Journal of neural engineering.

[52]  José J. Cañas,et al.  Saccadic peak velocity as an alternative index of operator attention: A short review , 2013 .

[53]  Joel R. Meyer,et al.  A large-scale distributed network for covert spatial attention: further anatomical delineation based on stringent behavioural and cognitive controls. , 1999, Brain : a journal of neurology.

[54]  Jie Lin,et al.  Towards Detection of Bus Driver Fatigue Based on Robust Visual Analysis of Eye State , 2017, IEEE Transactions on Intelligent Transportation Systems.

[55]  Natasha M. Maurits,et al.  Data-Driven Visualization and Group Analysis of Multichannel EEG Coherence with Functional Units , 2008, IEEE Transactions on Visualization and Computer Graphics.

[56]  M. Frese,et al.  Mental fatigue and the control of cognitive processes: effects on perseveration and planning. , 2003, Acta psychologica.

[57]  A. Nobre The attentive homunculus: Now you see it, now you don't , 2001, Neuroscience & Biobehavioral Reviews.

[58]  Guanghua Xu,et al.  Effects of Mental Load and Fatigue on Steady-State Evoked Potential Based Brain Computer Interface Tasks: A Comparison of Periodic Flickering and Motion-Reversal Based Visual Attention , 2016, PloS one.

[59]  A J Van Opstal,et al.  Blink-perturbed saccades in monkey. I. Behavioral analysis. , 2000, Journal of neurophysiology.

[60]  M. Härmä,et al.  The use of two-channel electro-oculography in automatic detection of unintentional sleep onset , 2007, Journal of Neuroscience Methods.

[61]  C. J. Stam,et al.  Cognition is related to resting-state small-world network topology: an magnetoencephalographic study , 2011, Neuroscience.

[62]  Thomas W. Calvert,et al.  A Systems Model of the Effects of Training on Physical Performance , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[63]  M M Mesulam,et al.  Large‐scale neurocognitive networks and distributed processing for attention, language, and memory , 1990, Annals of neurology.

[64]  Jürgen Schmidt,et al.  Eye blink detection for different driver states in conditionally automated driving and manual driving using EOG and a driver camera , 2017, Behavior Research Methods.

[65]  Jorge Moll,et al.  Structural and functional connectivity changes in response to short-term neurofeedback training with motor imagery , 2019, NeuroImage.

[66]  T A Ranney,et al.  Prolonged exposure to glare and driving time: effects on performance in a driving simulator. , 1999, Accident; analysis and prevention.

[67]  W. Qin,et al.  Interindividual reaction time variability is related to resting-state network topology: an electroencephalogram study , 2012, Neuroscience.

[68]  John J. B. Allen,et al.  Prelude to and Resolution of an Error: EEG Phase Synchrony Reveals Cognitive Control Dynamics during Action Monitoring , 2009, The Journal of Neuroscience.

[69]  W. R. Miles,et al.  Horizontal eye movements at the onset of sleep. , 1929 .

[70]  Richard M. Leahy,et al.  A note on the phase locking value and its properties , 2013, NeuroImage.

[71]  Maarten A. S. Boksem,et al.  Mental fatigue: Costs and benefits , 2008, Brain Research Reviews.

[72]  Maarten A. S. Boksem,et al.  Mental fatigue, motivation and action monitoring , 2006, Biological Psychology.

[73]  W. Dement,et al.  Quantification of sleepiness: a new approach. , 1973, Psychophysiology.

[74]  Olaf Sporns,et al.  Complex network measures of brain connectivity: Uses and interpretations , 2010, NeuroImage.

[75]  D. Schroeder,et al.  Blink Rate: A Possible Measure of Fatigue , 1994, Human factors.

[76]  Agostinho C. Rosa,et al.  Fatigue Evaluation Using Multi-Scale Entropy of EEG in SSVEP-Based BCI , 2019, IEEE Access.

[77]  A. Craig,et al.  Driver fatigue: electroencephalography and psychological assessment. , 2002, Psychophysiology.

[78]  O. Sporns,et al.  Complex brain networks: graph theoretical analysis of structural and functional systems , 2009, Nature Reviews Neuroscience.

[79]  Amin Mahnam,et al.  An Adaptive SSVEP-Based Brain-Computer Interface to Compensate Fatigue-Induced Decline of Performance in Practical Application , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[80]  Feng Wan,et al.  Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces , 2014, Biomedical engineering online.

[81]  Lawrence Stark,et al.  Overlapping saccades and glissades are produced by fatigue in the saccadic eye movement system , 1975, Experimental Neurology.

[82]  Manuel Schabus,et al.  A shift of visual spatial attention is selectively associated with human EEG alpha activity , 2005, The European journal of neuroscience.

[83]  Simon Hanslmayr,et al.  The role of alpha oscillations in temporal attention , 2011, Brain Research Reviews.

[84]  A. Fuchs,et al.  Lid-eye coordination during vertical gaze changes in man and monkey. , 1988, Journal of neurophysiology.

[85]  Anton Nijholt,et al.  Guest Editorial: Brain/neuronal - Computer game interfaces and interaction , 2013, IEEE Trans. Comput. Intell. AI Games.

[86]  Yangsong Zhang,et al.  Periodic Visual Stimulation Induces Resting-State Brain Network Reconfiguration , 2018, Front. Comput. Neurosci..

[87]  Peng Chen,et al.  Driving Fatigue Classification Based on Fusion Entropy Analysis Combining EOG and EEG , 2019, IEEE Access.

[88]  Alex R. Wade,et al.  Visual field maps and stimulus selectivity in human ventral occipital cortex , 2005, Nature Neuroscience.

[89]  Ying Du,et al.  Synchronization study in ring-like and grid-like neuronal networks , 2011, Cognitive Neurodynamics.

[90]  T. M. Nelson,et al.  Development of fatigue symptoms during simulated driving. , 1997, Accident; analysis and prevention.

[91]  Hristos S. Courellis,et al.  Using a Novel Approach to Assess Dynamic Cortical Connectivity Changes Following Neurofeedback Training in Children on the Autism Spectrum , 2019, Neurotechnology and Brain Stimulation in Pediatric Psychiatric and Neurodevelopmental Disorders.

[92]  Beatriz Rey,et al.  Power Spectral Density and Functional Connectivity Changes due to a Sensorimotor Neurofeedback Training: A Preliminary Study , 2019, Neural plasticity.

[93]  Maarten A. S. Boksem,et al.  Effects of mental fatigue on attention: an ERP study. , 2005, Brain research. Cognitive brain research.

[94]  Gabriela Castellano,et al.  EEG Functional Connectivity Patterns Over the Course of Neurofeedback Attention Training for Healthy Subjects: A Pilot Study , 2019, XXVI Brazilian Congress on Biomedical Engineering.

[95]  Guanghua Xu,et al.  The Role of Visual Noise in Influencing Mental Load and Fatigue in a Steady-State Motion Visual Evoked Potential-Based Brain-Computer Interface , 2017, Sensors.

[96]  Gernot R. Müller-Putz,et al.  Effects of mental workload and fatigue on the P300, alpha and theta band power during operation of an ERP (P300) brain–computer interface , 2014, Biological Psychology.

[97]  Robertas Damasevicius,et al.  Gamification of Eye Exercises for Evaluating Eye Fatigue , 2019, International Conference on Artificial Intelligence and Soft Computing.

[98]  J. Roerdink,et al.  High-density EEG coherence analysis using functional units applied to mental fatigue , 2008, Journal of Neuroscience Methods.

[99]  P. König,et al.  Top-down processing mediated by interareal synchronization. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[100]  J. Fell,et al.  The role of phase synchronization in memory processes , 2011, Nature Reviews Neuroscience.

[101]  Hong Wang,et al.  Real-Time ECG-Based Detection of Fatigue Driving Using Sample Entropy , 2018, Entropy.

[102]  Mauro Ursino,et al.  Detection of sleep onset by analysis of slow eye movements: a preliminary study of MSLT recordings. , 2009, Sleep medicine.

[103]  Michael Frese,et al.  The Impact of Mental Fatigue on Exploration in a Complex Computer Task: Rigidity and Loss of Systematic Strategies , 2003, Hum. Factors.

[104]  Gerhard Tröster,et al.  Eye Movement Analysis for Activity Recognition Using Electrooculography , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[105]  Yvonne Tran,et al.  The psychophysiological determinants of fatigue. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[106]  R. Kahn,et al.  Efficiency of Functional Brain Networks and Intellectual Performance , 2009, The Journal of Neuroscience.

[107]  C. Ciacci,et al.  Fatigue in adult coeliac disease , 2005, Alimentary pharmacology & therapeutics.

[108]  A Sprenger,et al.  Effects of voluntary blinks on saccades, vergence eye movements, and saccade-vergence interactions in humans. , 2002, Journal of neurophysiology.

[109]  Jens Christian Claussen,et al.  The phase response of the cortical slow oscillation , 2012, Cognitive Neurodynamics.

[110]  H. Okamura,et al.  Development and validation of the cancer fatigue scale: a brief, three-dimensional, self-rating scale for assessment of fatigue in cancer patients. , 2000, Journal of pain and symptom management.

[111]  Sanjeev Malik,et al.  Visual analog scale (VAS) for assessment of acute mountain sickness (AMS) on Aconcagua. , 2011, Wilderness & environmental medicine.

[112]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[113]  J. Santamaria,et al.  The EEG of Drowsiness in Normal Adults , 1987, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[114]  Jiankun Hu,et al.  Convolutional Neural Networks Using Dynamic Functional Connectivity for EEG-Based Person Identification in Diverse Human States , 2019, IEEE Transactions on Information Forensics and Security.

[115]  Todd B. Parrish,et al.  The posterior cingulate and medial prefrontal cortex mediate the anticipatory allocation of spatial attention , 2003, NeuroImage.

[116]  Robert Du,et al.  Towards measurement of brain function in operational environments , 1995, Biological Psychology.

[117]  C. Scheib,et al.  Brainstem Influence on Thalamocortical Oscillations during Anesthesia Emergence , 2017, Front. Syst. Neurosci..

[118]  R Broughton,et al.  Sensitivity of the stanford sleepiness scale to the effects of cumulative partial sleep deprivation and recovery oversleeping. , 1981, Sleep.

[119]  P. Fries A mechanism for cognitive dynamics: neuronal communication through neuronal coherence , 2005, Trends in Cognitive Sciences.

[120]  Christa Neuper,et al.  Rehabilitation with Brain-Computer Interface Systems , 2008, Computer.

[121]  Mark D'Esposito,et al.  Alpha-Band Phase Synchrony Is Related to Activity in the Fronto-Parietal Adaptive Control Network , 2012, The Journal of Neuroscience.

[122]  Byron J. Gajewski,et al.  Reliability, Responsiveness, and Validity of the Visual Analog Fatigue Scale to Measure Exertion Fatigue in People with Chronic Stroke: A Preliminary Study , 2010, Stroke research and treatment.

[123]  Jan-Mathijs Schoffelen,et al.  A Tutorial Review of Functional Connectivity Analysis Methods and Their Interpretational Pitfalls , 2016, Front. Syst. Neurosci..

[124]  A. Nobre,et al.  Orienting Attention to Locations in Internal Representations , 2003, Journal of Cognitive Neuroscience.

[125]  Cornelis J Stam,et al.  Graph theoretical analysis of complex networks in the brain , 2007, Nonlinear biomedical physics.

[126]  Ronald M. Aarts,et al.  A Survey of Stimulation Methods Used in SSVEP-Based BCIs , 2010, Comput. Intell. Neurosci..

[127]  A. Pikovsky,et al.  Synchronization: Theory and Application , 2003 .

[128]  Yangsong Zhang,et al.  SSVEP Response Is Related to Functional Brain Network Topology Entrained by the Flickering Stimulus , 2013, PloS one.

[129]  Feng Wan,et al.  State-of-the-art non-invasive brain–computer interface for neural rehabilitation: A review , 2020 .

[130]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[131]  J. Stern,et al.  Blinks as an index of cognitive activity during reading , 1991, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[132]  Benjamin Blankertz,et al.  EEG predictors of covert vigilant attention , 2014, Journal of neural engineering.

[133]  T. Chau,et al.  Effects of user mental state on EEG-BCI performance , 2015, Front. Hum. Neurosci..

[134]  P. Suffczynski,et al.  On the Quantification of SSVEP Frequency Responses in Human EEG in Realistic BCI Conditions , 2013, PloS one.

[135]  Bao-Liang Lu,et al.  An EOG-based Vigilance Estimation Method Applied for Driver Fatigue Detection , 2015 .

[136]  Markus Amann,et al.  Opioid‐mediated muscle afferents inhibit central motor drive and limit peripheral muscle fatigue development in humans , 2009, The Journal of physiology.

[137]  Paul Eling,et al.  Mental fatigue disturbs local processing more than global processing , 2006, Psychological research.

[138]  Rajesh P. N. Rao,et al.  Towards adaptive classification for BCI , 2006, Journal of neural engineering.

[139]  T. Åkerstedt,et al.  Subjective and objective sleepiness in the active individual. , 1990, The International journal of neuroscience.