EEG-Based Cognitive Control Behaviour Assessment: an Ecological study with Professional Air Traffic Controllers

Several models defining different types of cognitive human behaviour are available. For this work, we have selected the Skill, Rule and Knowledge (SRK) model proposed by Rasmussen in 1983. This model is currently broadly used in safety critical domains, such as the aviation. Nowadays, there are no tools able to assess at which level of cognitive control the operator is dealing with the considered task, that is if he/she is performing the task as an automated routine (skill level), as procedures-based activity (rule level), or as a problem-solving process (knowledge level). Several studies tried to model the SRK behaviours from a Human Factor perspective. Despite such studies, there are no evidences in which such behaviours have been evaluated from a neurophysiological point of view, for example, by considering brain activity variations across the different SRK levels. Therefore, the proposed study aimed to investigate the use of neurophysiological signals to assess the cognitive control behaviours accordingly to the SRK taxonomy. The results of the study, performed on 37 professional Air Traffic Controllers, demonstrated that specific brain features could characterize and discriminate the different SRK levels, therefore enabling an objective assessment of the degree of cognitive control behaviours in realistic settings.

[1]  Wolfgang Klimesch,et al.  The Structure of Long-term Memory: A Connectivity Model of Semantic Processing , 1996 .

[2]  Jan Friso Groote,et al.  20 Years After , 2007 .

[3]  Thomas D. Y. Chung,et al.  A Simple Statistical Parameter for Use in Evaluation and Validation of High Throughput Screening Assays , 1999, Journal of biomolecular screening.

[4]  Laura Astolfi,et al.  Time varying effective connectivity for describing brain network changes induced by a memory rehabilitation treatment , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[5]  Fabio Babiloni,et al.  Air-traffic-controllers (ATCO): neurophysiological analysis of training and workload , 2015 .

[6]  S Pozzi,et al.  A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. , 2016, Progress in brain research.

[7]  G. Tononi.  Consciousness as Integrated Information: a Provisional Manifesto , 2008, The Biological Bulletin.

[8]  L. Giraudet,et al.  The neuroergonomic evaluation of human machine interface design in air traffic control using behavioral and EEG/ERP measures , 2015, Behavioural Brain Research.

[9]  C. Summerfield,et al.  Expectation (and attention) in visual cognition , 2009, Trends in Cognitive Sciences.

[10]  E. Adrian,et al.  THE BERGER RHYTHM: POTENTIAL CHANGES FROM THE OCCIPITAL LOBES IN MAN , 1934 .

[11]  R. T. Kellogg.  Is conscious attention necessary for long-term storage? , 1980 .

[12]  F. Crick.  Function of the thalamic reticular complex: the searchlight hypothesis. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.

[14]  Wolfgang Klimesch,et al.  Evoked alpha and early access to the knowledge system: The P1 inhibition timing hypothesis , 2011, Brain Research.

[15]  Fabio Babiloni,et al.  A neurophysiological training evaluation metric for Air Traffic Management , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[17]  Fabio Babiloni,et al.  Reliability over time of EEG-based mental workload evaluation during Air Traffic Management (ATM) tasks , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[18]  G. Schwartz,et al.  Consciousness and Self-Regulation , 1976 .

[19]  Jean-Paul Imbert,et al.  Think different: how we completely changed the visualization of pseudo-pilot , 2015, Graphics Interface.

[20]  Paolo Fiorini,et al.  Neurophysiological measures for users' training objective assessment during simulated robot-assisted laparoscopic surgery , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[21]  John Gruzelier,et al.  A theory of alpha/theta neurofeedback, creative performance enhancement, long distance functional connectivity and psychological integration , 2009, Cognitive Processing.

[22]  M. Sterman,et al.  Multiband topographic EEG analysis of a simulated visuomotor aviation task. , 1994, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[23]  Fabio Babiloni,et al.  Training-induced changes in information transfer efficiency of the brain network: A functional connectome approach , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[24]  M. Posner,et al.  The attention system of the human brain. , 1990, Annual review of neuroscience.

[25]  Brendan Z. Allison,et al.  Is It Significant? Guidelines for Reporting BCI Performance , 2012 .

[26]  W. Klimesch,et al.  EEG alpha oscillations: The inhibition–timing hypothesis , 2007, Brain Research Reviews.

[27]  Laura Astolfi,et al.  Investigating Cooperative Behavior in Ecological Settings: An EEG Hyperscanning Study , 2016, PloS one.

[28]  Laura Astolfi,et al.  Frontal EEG theta changes assess the training improvements of novices in flight simulation tasks , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[29]  G. Pfurtscheller,et al.  Event-related dynamics of cortical rhythms: frequency-specific features and functional correlates. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[31]  Manuel Schabus,et al.  Fronto-parietal EEG coherence in theta and upper alpha reflect central executive functions of working memory. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[32]  N. McGlynn.  Thinking fast and slow. , 2014, Australian veterinary journal.

[33]  J. Reason.  Human error: models and management , 2000, BMJ : British Medical Journal.

[34]  T. Egner,et al.  Validating the efficacy of neurofeedback for optimising performance. , 2006, Progress in brain research.

[35]  Richard A. Schmidt,et al.  Optimizing summary knowledge of results for skill learning , 1990 .

[36]  Fabio Babiloni,et al.  Avionic technology testing by using a cognitive neurometric index: A study with professional helicopter pilots , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[37]  M. Benedek,et al.  Tackling creativity at its roots: Evidence for different patterns of EEG alpha activity related to convergent and divergent modes of task processing , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[38]  J. Rassmusen,et al.  Information Processing and Human - Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[39]  Heather Shearer,et al.  The Human Contribution: Unsafe Acts, Accidents and Heroic Recoveries , 2009 .

[40]  A. Marcel.  Conscious and unconscious perception: Experiments on visual masking and word recognition , 1983, Cognitive Psychology.

[41]  W. Klimesch.  EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[42]  Catherine M. Burns,et al.  Ecological Interface Design , 2004 .

[43]  D. LaBerge,et al.  Attention, consciousness, and electrical wave activity within the cortical column. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[44]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[45]  J. Zauszniewski,et al.  Methodological triangulation: an approach to understanding data. , 2012, Nurse researcher.

[46]  Fabio Babiloni,et al.  On the Use of Cognitive Neurometric Indexes in Aeronautic and Air Traffic Management Environments , 2015, Symbiotic.

[47]  Rene J. de Pontbriand,et al.  Modeling Human Behavior with Integrated Cognitive Architectures: Comparison, Evaluation, and Validation Edited by Kevin A. Gluck & Richard A. Pew 2005, 440 pages, $55.00 Mahwah, NJ: Lawrence Erlbaum Associates ISBN 0-8058-5048-1 , 2007 .

[48]  G. Kochanska,et al.  Effortful control in early childhood: continuity and change, antecedents, and implications for social development. , 2000, Developmental psychology.

[49]  Stéphane Bonnet,et al.  Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[50]  Fabio Babiloni,et al.  Investigation of the effect of EEG-BCI on the simultaneous execution of flight simulation and attentional tasks , 2015, Medical & Biological Engineering & Computing.

[51]  Kim J. Vicente,et al.  Coping with Human Errors through System Design: Implications for Ecological Interface Design , 1989, Int. J. Man Mach. Stud..

[52]  Kim J. Vicente,et al.  Ecological interface design: theoretical foundations , 1992, IEEE Trans. Syst. Man Cybern..

[53]  J. Kaiser,et al.  Human gamma-frequency oscillations associated with attention and memory , 2007, Trends in Neurosciences.

[54]  Cenap Çakmak.  Review and Analysis , 2017 .

[55]  S. Petersen,et al.  The effects of practice on the functional anatomy of task performance. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[56]  F C Hoppensteadt,et al.  The searchlight hypothesis , 1991, Journal of mathematical biology.

[57]  S. Petersen,et al.  Contributions of the pulvinar to visual spatial attention , 1987, Neuropsychologia.

[58]  E Donchin,et al.  A new method for off-line removal of ocular artifact. , 1983, Electroencephalography and clinical neurophysiology.

[59]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[60]  Fabio Babiloni,et al.  An electroencephalographic Peak Density Function to detect memorization during the observation of TV commercials , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[61]  J. Buhle,et al.  Typologies of attentional networks , 2006, Nature Reviews Neuroscience.

[62]  Anna Papst,et al.  Fast and Slow , 2008, Science.

[63]  Karim Jerbi,et al.  Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy , 2015, Journal of Neuroscience Methods.

[64]  G A Orban,et al.  Attention-dependent suppression of metabolic activity in the early stages of the macaque visual system. , 2000, Cerebral cortex.

[65]  M. Posner,et al.  The attention system of the human brain: 20 years after. , 2012, Annual review of neuroscience.

[66]  R. VanRullen,et al.  An oscillatory mechanism for prioritizing salient unattended stimuli , 2012, Trends in Cognitive Sciences.

[67]  Raja Parasuraman,et al.  Neuroergonomics: The Brain at Work , 2006 .

[68]  Dan Kemp.  Of Models and Management , 2018 .

[69]  Brian Hilburn,et al.  COGNITIVE COMPLEXITY IN AIR TRAFFIC CONTROL: A LITERATURE REVIEW , 2004 .

[70]  Febo Cincotti,et al.  Towards a multimodal bioelectrical framework for the online mental workload evaluation , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[71]  Ole Jensen,et al.  Alpha Oscillations Correlate with the Successful Inhibition of Unattended Stimuli , 2011, Journal of Cognitive Neuroscience.

[72]  A. Compston.  The Berger rhythm: potential changes from the occipital lobes in man. , 2010, Brain : a journal of neurology.

[73]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[74]  D. Bamber.  The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .

[75]  Laura Astolfi,et al.  Assessment of mental fatigue during car driving by using high resolution EEG activity and neurophysiologic indices , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[76]  Fabio Babiloni,et al.  Evaluation of the workload and drowsiness during car driving by using high resolution EEG activity and neurophysiologic indices , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[77]  Donald A. Norman,et al.  Attention to Action , 1986 .

[78]  C. Neuper,et al.  Event-related dynamics of brain oscillations , 2006 .

[79]  Fabio Babiloni,et al.  Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment , 2016, Front. Hum. Neurosci..

[80]  M. Desseilles,et al.  Both the Hippocampus and Striatum Are Involved in Consolidation of Motor Sequence Memory , 2008, Neuron.

[81]  A. Zeman.  Attentional Processing. The Brain's Art of Mindfulness , 1996 .

[82]  R. B. Freeman,et al.  Cognition and Motor Processes , 1983 .

[83]  Fabio Babiloni,et al.  Investigating Driver Fatigue versus Alertness Using the Granger Causality Network , 2015, Sensors.

[84]  H. ThalerRichard,et al.  Theoretical foundations , 2007, Interpersonal Meaning in Multimodal English Textbooks.

[85]  J. Bhattacharya,et al.  Deconstructing Insight: EEG Correlates of Insightful Problem Solving , 2008, PloS one.

[86]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[87]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[88]  M. Benedek,et al.  EEG alpha synchronization is related to top-down processing in convergent and divergent thinking , 2011, Neuropsychologia.

[89]  M. Pinsk,et al.  Attention modulates responses in the human lateral geniculate nucleus , 2002, Nature Neuroscience.

[90]  Glenn F. Wilson,et al.  Psychophysiological responses to changes in workload during simulated air traffic control , 1996, Biological Psychology.

[91]  Theodore P. Zanto,et al.  Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory , 2011, Nature Neuroscience.

[92]  G. Tononi.  An information integration theory of consciousness , 2004, BMC Neuroscience.

[93]  Fabio Babiloni,et al.  Quantitative Assessment of the Training Improvement in a Motor-Cognitive Task by Using EEG, ECG and EOG Signals , 2015, Brain Topography.

[94]  Fabio Babiloni,et al.  A new regression-based method for the eye blinks artifacts correction in the EEG signal, without using any EOG channel , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[95]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[96]  I. Timofeev,et al.  Sleep Oscillations in the Thalamocortical System Induce Long-Term Neuronal Plasticity , 2012, Neuron.

[97]  J. Kropotov.  Quantitative EEG, Event-Related Potentials and Neurotherapy , 2008 .

[98]  Jens Rasmussen,et al.  The Definition of Human Error and a Taxonomy for Technical System Design , 1987 .

[99]  Jean-Paul Imbert,et al.  The Role of Cognitive and Perceptual Loads in Inattentional Deafness , 2016, Front. Hum. Neurosci..

[100]  G. G. Molina.  Towards a Practical Brain-Computer Interface , 2007 .

[101]  R. Elul.  Gaussian Behavior of the Electroencephalogram: Changes during Performance of Mental Task , 1969, Science.

[102]  Karl J. Friston.  The free-energy principle: a rough guide to the brain? , 2009, Trends in Cognitive Sciences.

[103]  W. Klimesch.  Alpha-band oscillations, attention, and controlled access to stored information , 2012, Trends in Cognitive Sciences.

[104]  Marijke A. Beulen,et al.  The role of theta oscillations in memory and decision making , 2011 .

[105]  Bernard J. Baars,et al.  Some Essential Differences between Consciousness and Attention, Perception, and Working Memory , 1997, Consciousness and Cognition.

[106]  D. LaBerge,et al.  Positron emission tomographic measurements of pulvinar activity during an attention task , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.