Biomarker-Informed Machine Learning Model of Cognitive Fatigue from a Heart Rate Response Perspective

Cognitive fatigue is a psychological state characterised by feelings of tiredness and impaired cognitive functioning arising from high cognitive demands. This paper examines the recent research progress on the assessment of cognitive fatigue and provides informed recommendations for future research. Traditionally, cognitive fatigue is introspectively assessed through self-report or objectively inferred from a decline in behavioural performance. However, more recently, researchers have attempted to explore the biological underpinnings of cognitive fatigue to understand and measure this phenomenon. In particular, there is evidence indicating that the imbalance between sympathetic and parasympathetic nervous activity appears to be a physiological correlate of cognitive fatigue. This imbalance has been indexed through various heart rate variability indices that have also been proposed as putative biomarkers of cognitive fatigue. Moreover, in contrast to traditional inferential methods, there is also a growing research interest in using data-driven approaches to assessing cognitive fatigue. The ubiquity of wearables with the capability to collect large amounts of physiological data appears to be a major facilitator in the growth of data-driven research in this area. Preliminary findings indicate that such large datasets can be used to accurately predict cognitive fatigue through various machine learning approaches. Overall, the potential of combining domain-specific knowledge gained from biomarker research with machine learning approaches should be further explored to build more robust predictive models of cognitive fatigue.

[1]  P. Melillo,et al.  Nonlinear Heart Rate Variability features for real-life stress detection. Case study: students under stress due to university examination , 2011, Biomedical engineering online.

[2]  Miriam A. M. Capretz,et al.  Machine Learning With Big Data: Challenges and Approaches , 2017, IEEE Access.

[3]  I D Brown,et al.  Prospects for technological countermeasures against driver fatigue. , 1997, Accident; analysis and prevention.

[4]  G. Wylie,et al.  The relationship between outcome prediction and cognitive fatigue: A convergence of paradigms , 2017, Cognitive, affective & behavioral neuroscience.

[5]  L. Fleisher,et al.  Thermoregulation and heart rate variability. , 1996, Clinical science.

[6]  Alexei Botchkarev,et al.  A New Typology Design of Performance Metrics to Measure Errors in Machine Learning Regression Algorithms , 2019, Interdisciplinary Journal of Information, Knowledge, and Management.

[7]  Ross E. G. Upshur,et al.  Three Problems with Big Data and Artificial Intelligence in Medicine , 2019, Perspectives in biology and medicine.

[8]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[9]  Ke Wang,et al.  The digital biomarker discovery pipeline: An open-source software platform for the development of digital biomarkers using mHealth and wearables data , 2020, Journal of Clinical and Translational Science.

[10]  Osamu Mizuno,et al.  Predicting Changes in Cognitive Performance Using Heart Rate Variability , 2017, IEICE Trans. Inf. Syst..

[11]  D. Adam,et al.  Assessment of autonomic function in humans by heart rate spectral analysis. , 1985, The American journal of physiology.

[12]  X. Bornas,et al.  Non-linear Heart Rate Variability as a Discriminator of Internalizing Psychopathology and Negative Affect in Children With Internalizing Problems and Healthy Controls , 2018, Front. Physiol..

[13]  Peter Newman,et al.  The Influence of Natural Sounds on Attention Restoration , 2016 .

[14]  Chong Zhang,et al.  Estimation of the cortical functional connectivity by directed transfer function during mental fatigue. , 2010, Applied ergonomics.

[15]  L. Rönnbäck,et al.  A self-assessment questionnaire for mental fatigue and related symptoms after neurological disorders and injuries , 2010, Brain injury.

[16]  E. Olivier,et al.  Dissociation between mental fatigue and motivational state during prolonged mental activity , 2015, Front. Behav. Neurosci..

[17]  M. AlGhatrif,et al.  A brief review: history to understand fundamentals of electrocardiography , 2012, Journal of community hospital internal medicine perspectives.

[18]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[19]  Dhruv R. Seshadri,et al.  Wearable sensors for monitoring the physiological and biochemical profile of the athlete , 2019, npj Digital Medicine.

[20]  Ming Chen,et al.  Spectral analysis of heart rate variability as a quantitative indicator of driver mental fatigue , 2002 .

[21]  Leandro Pecchia,et al.  Heart Rate Variability (HRV) Analysis: A Methodology for Organizational Neuroscience , 2019 .

[22]  Eman M. G. Younis,et al.  Designing and evaluating mobile self-reporting techniques: crowdsourcing for citizen science , 2019, Personal and Ubiquitous Computing.

[23]  J. Ridley Studies of Interference in Serial Verbal Reactions , 2001 .

[24]  I. Winkler,et al.  Short-term cognitive fatigue effect on auditory temporal order judgments , 2020, Experimental Brain Research.

[25]  Q. P. He,et al.  Application of Systems Engineering Principles and Techniques in Biological Big Data Analytics: A Review , 2020, Processes.

[26]  Kouzi Yamaguti,et al.  Autonomic nervous alterations associated with daily level of fatigue , 2011, Behavioral and Brain Functions.

[27]  Oleg S. Pianykh,et al.  Current Applications and Future Impact of Machine Learning in Radiology. , 2018, Radiology.

[28]  Seiki Tajima,et al.  Central nervous system fatigue alters autonomic nerve activity. , 2009, Life sciences.

[29]  Adam Fletcher,et al.  Simulated train driving: fatigue, self-awareness and cognitive disengagement. , 2007, Applied ergonomics.

[30]  Masaaki Tanaka,et al.  Neural effects of mental fatigue caused by continuous attention load: A magnetoencephalography study , 2014, Brain Research.

[31]  J. Roerdink,et al.  The influence of mental fatigue and motivation on neural network dynamics; an EEG coherence study , 2009, Brain Research.

[32]  Susana Rubio-Valdehita,et al.  Psychological Capital, Workload, and Burnout: What’s New? The Impact of Personal Accomplishment to Promote Sustainable Working Conditions , 2020, Sustainability.

[33]  Feng Zhou,et al.  Driver fatigue transition prediction in highly automated driving using physiological features , 2020, Expert Syst. Appl..

[34]  R. Heinzer,et al.  [Driver fatigue]. , 2007, Revue medicale suisse.

[35]  Louise Venables,et al.  The influence of task demand and learning on the psychophysiological response. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[36]  S. Anwer,et al.  Evaluation of Physiological Metrics as Real-Time Measurement of Physical Fatigue in Construction Workers: State-of-the-Art Review , 2021 .

[37]  Giulio Gabrieli,et al.  PySiology: A Python Package for Physiological Feature Extraction , 2020, Neural Approaches to Dynamics of Signal Exchanges.

[38]  S. Huffel,et al.  Instantaneous changes in heart rate regulation due to mental load in simulated office work , 2011, European Journal of Applied Physiology.

[39]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[40]  Chong Zhang,et al.  Automatic recognition of cognitive fatigue from physiological indices by using wavelet packet transform and kernel learning algorithms , 2009, Expert Syst. Appl..

[41]  J. R. Simon,et al.  CHOICE REACTION TIME AS A FUNCTION OF ANGULAR STIMULUS-RESPONSE CORRESPONDENCE AND AGE , 1963 .

[42]  Peter A. Hancock,et al.  ACTIVE AND PASSIVE FATIGUE STATES , 2001 .

[43]  Weiqiang Zhang,et al.  Detection of mental fatigue state with wearable ECG devices , 2018, Int. J. Medical Informatics.

[44]  G. Howard,et al.  Self-report: psychology's four-letter word. , 2010, American Journal of Psychology.

[45]  M. Toichi,et al.  A new method of assessing cardiac autonomic function and its comparison with spectral analysis and coefficient of variation of R-R interval. , 1997, Journal of the autonomic nervous system.

[46]  J. Saul,et al.  Beat-to-Beat Variations of Heart Rate Reflect Modulation of Cardiac Autonomic Outflow , 1990 .

[47]  A. Buchner,et al.  Drivers' misjudgement of vigilance state during prolonged monotonous daytime driving. , 2009, Accident; analysis and prevention.

[48]  T. Yarkoni,et al.  Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning , 2017, Perspectives on psychological science : a journal of the Association for Psychological Science.

[49]  Majid A. Al-Taee,et al.  HRV-based operator fatigue analysis and classification using wearable sensors , 2016, 2016 13th International Multi-Conference on Systems, Signals & Devices (SSD).

[50]  E. Marengo,et al.  Biomarkers Discovery through Multivariate Statistical Methods: A Review of Recently Developed Methods and Applications in Proteomics , 2014 .

[51]  D. Bonaduce,et al.  Effects of Converting Enzyme Inhibition on Heart Period Variability in Patients With Acute Myocardial Infarction , 1994, Circulation.

[52]  José Hernández-Orallo,et al.  An experimental comparison of performance measures for classification , 2009, Pattern Recognit. Lett..

[53]  W. Kibbe,et al.  Investigating sources of inaccuracy in wearable optical heart rate sensors , 2020, npj Digital Medicine.

[54]  S. Porges The polyvagal perspective , 2007, Biological Psychology.

[55]  P. Enck,et al.  Heart rate variability biofeedback therapy and graded exercise training in management of chronic fatigue syndrome: An exploratory pilot study. , 2017, Journal of psychosomatic research.

[56]  S. Perrey,et al.  Decrease in heart rate variability with overtraining: assessment by the Poincaré plot analysis , 2004, Clinical physiology and functional imaging.

[57]  J. Goldberger,et al.  Sympathovagal balance: how should we measure it? , 1999, The American journal of physiology.

[58]  R. Tortelli,et al.  The use of wearable/portable digital sensors in Huntington's disease: A systematic review , 2021, Parkinsonism & related disorders.

[59]  R. Job,et al.  Defining fatigue as a condition of the organism and distinguishing it from habituation, adaptation, and boredom , 2001 .

[60]  Roee Holtzer,et al.  Cognitive Fatigue Defined in the Context of Attention Networks , 2010, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[61]  R. Lane,et al.  A model of neurovisceral integration in emotion regulation and dysregulation. , 2000, Journal of affective disorders.

[62]  J L Kenemans,et al.  Mental Fatigue : Costs and Benefits , 2005 .

[63]  Cinna Soltanpur,et al.  A review on wearable photoplethysmography sensors and their potential future applications in health care , 2018, International journal of biosensors & bioelectronics.

[64]  T. Chai,et al.  Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature , 2014 .

[65]  Kar Fye Alvin Lee,et al.  The effects of anxiety, depressive, and obsessive-compulsive subclinical symptoms on performance monitoring. , 2020, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[66]  G. Clifford,et al.  A review of physiological and behavioral monitoring with digital sensors for neuropsychiatric illnesses , 2018, Physiological measurement.

[67]  N. Ishii,et al.  [The relationship between self-awareness of fatigue symptoms and working conditions in female nurses]. , 2015, Sangyo eiseigaku zasshi = Journal of occupational health.

[68]  D. Goldstein,et al.  Low frequency power of heart rate variability reflects baroreflex function, not cardiac sympathetic innervation , 2011, Clinical Autonomic Research.

[69]  J. Thayer,et al.  Vagal influence on working memory and attention. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[70]  Jaques Reifman,et al.  Application of Information Technology: A Method for Automatic Identification of Reliable Heart Rates Calculated from ECG and PPG Waveforms , 2006, J. Am. Medical Informatics Assoc..

[71]  Jeffrey H Goode,et al.  Are pilots at risk of accidents due to fatigue? , 2003, Journal of safety research.

[72]  Gaël Varoquaux,et al.  Predictive models avoid excessive reductionism in cognitive neuroimaging , 2019, Current Opinion in Neurobiology.

[73]  Gavriel Salvendy,et al.  Prediction of Mental Workload in Single and Multiple Tasks Environments , 2000 .

[74]  Malini Subramanyam,et al.  Mental Workload and Cognitive Fatigue: A Study , 2013 .

[75]  P. Stein,et al.  Heart Rate Variability: Measurement and Clinical Utility , 2005, Annals of noninvasive electrocardiology : the official journal of the International Society for Holter and Noninvasive Electrocardiology, Inc.

[76]  Chong Zhang,et al.  Estimating mental fatigue based on electroencephalogram and heart rate variability , 2010 .

[77]  Iztok Fister,et al.  Sensors and Functionalities of Non-Invasive Wrist-Wearable Devices: A Review , 2018, Sensors.

[78]  Gary N Smith,et al.  The paradox of big data , 2020, SN Applied Sciences.

[79]  M. Frese,et al.  Mental fatigue and the control of cognitive processes: effects on perseveration and planning. , 2003, Acta psychologica.

[80]  E. L. Fisk,et al.  Fatigue in Industry. , 1922, American journal of public health.

[81]  T. Seppänen,et al.  Quantitative beat-to-beat analysis of heart rate dynamics during exercise. , 1996, The American journal of physiology.

[82]  S. Wessely,et al.  Development of a fatigue scale. , 1993, Journal of psychosomatic research.

[83]  Eduardo E. Benarroch,et al.  Central Autonomic Control , 2004 .

[84]  Alexei Botchkarev,et al.  Evaluating Performance of Regression Machine Learning Models Using Multiple Error Metrics in Azure Machine Learning Studio , 2018 .

[85]  Roger M Enoka,et al.  Fatigue and fatigability in neurologic illnesses , 2013, Neurology.

[86]  A. Meyer-Lindenberg,et al.  Machine Learning for Precision Psychiatry: Opportunities and Challenges. , 2017, Biological psychiatry. Cognitive neuroscience and neuroimaging.

[87]  N Egelund,et al.  Spectral analysis of heart rate variability as an indicator of driver fatigue. , 1982, Ergonomics.

[88]  Jessilyn Dunn,et al.  Wearables and the medical revolution. , 2018, Personalized medicine.

[89]  L. McCorry Physiology of the autonomic nervous system. , 2007, American journal of pharmaceutical education.

[90]  P. Falkai,et al.  Machine Learning Approaches for Clinical Psychology and Psychiatry. , 2018, Annual review of clinical psychology.

[91]  Mirja A. Peltola Role of Editing of R–R Intervals in the Analysis of Heart Rate Variability , 2011, Front. Physio..

[92]  Luke J. Chang,et al.  Building better biomarkers: brain models in translational neuroimaging , 2017, Nature Neuroscience.

[93]  M. Turiel,et al.  Power Spectral Analysis of Heart Rate and Arterial Pressure Variabilities as a Marker of Sympatho‐Vagal Interaction in Man and Conscious Dog , 1986, Circulation research.

[94]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[95]  D. Dinges,et al.  Sleep Deprivation and Vigilant Attention , 2008, Annals of the New York Academy of Sciences.

[96]  Rifai Chai,et al.  The influence of mental fatigue on brain activity: Evidence from a systematic review with meta-analyses. , 2020, Psychophysiology.

[97]  L. Kappos,et al.  The Fatigue Scale for Motor and Cognitive Functions (FSMC): validation of a new instrument to assess multiple sclerosis-related fatigue , 2009, Multiple sclerosis.

[98]  I. Gottesman,et al.  Where Do We Stand in the Quest for Neuropsychiatric Biomarkers and Endophenotypes and What Next , 2009 .

[99]  Zhihua Xiao Statistics and Data Mining , 2000 .

[100]  Sarah E. Burke,et al.  Compensatory Neural Responses to Cognitive Fatigue in Young and Older Adults , 2019, Front. Neural Circuits.

[101]  Ken Kelley,et al.  Big Data in Psychology: A Framework for Research Advancement , 2018, The American psychologist.

[102]  Aric A. Prather,et al.  More than a feeling: A unified view of stress measurement for population science , 2018, Frontiers in Neuroendocrinology.

[103]  D. DeMets,et al.  Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework , 2001, Clinical pharmacology and therapeutics.

[104]  Davide Ballabio,et al.  Multivariate comparison of classification performance measures , 2017 .

[105]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[106]  George I. Christopoulos,et al.  Activity Tracker–Based Metrics as Digital Markers of Cardiometabolic Health in Working Adults: Cross-Sectional Study , 2020, JMIR mHealth and uHealth.

[107]  D L Eckberg,et al.  Mechanisms underlying very-low-frequency RR-interval oscillations in humans. , 1998, Circulation.

[108]  M. Ding,et al.  Compensatory Neural Activity in Response to Cognitive Fatigue , 2016, The Journal of Neuroscience.

[109]  Heart Rate Variability Standards of Measurement , Physiological Interpretation , and Clinical Use Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology , 2022 .

[110]  Ron H. J. Scholte,et al.  Validity of the Empatica E4 Wristband to Measure Heart Rate Variability (HRV) Parameters: a Comparison to Electrocardiography (ECG) , 2020, Journal of Medical Systems.

[111]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[112]  E. Benarroch The central autonomic network: functional organization, dysfunction, and perspective. , 1993, Mayo Clinic proceedings.

[113]  Al-Baraa Abdulrahman Al-Mekhlafi,et al.  THE RELATIONSHIP BETWEEN FATIGUE AND DRIVING PERFORMANCE: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH , 2020 .

[114]  M. Mcdermott,et al.  Cognitive fatigue during a test requiring sustained attention: a pilot study , 2003, Multiple sclerosis.

[115]  Alex Lloyd,et al.  A comparison of methods used for inducing mental fatigue in performance research: individualised, dual-task and short duration cognitive tests are most effective , 2020, Ergonomics.

[116]  D. J. Mascord,et al.  Behavioral and physiological indices of fatigue in a visual tracking task , 1992 .

[117]  Kenshi Sakai,et al.  Human photoplethysmogram: new insight into chaotic characteristics , 2015 .

[118]  K. Chon,et al.  Can Photoplethysmography Variability Serve as an Alternative Approach to Obtain Heart Rate Variability Information? , 2008, Journal of Clinical Monitoring and Computing.

[119]  Udo Trutschel,et al.  Heart rate measures reflect the interaction of low mental workload and fatigue during driving simulation , 2012, AutomotiveUI.

[120]  Masaaki Tanaka,et al.  Effects of Mental Fatigue on Brain Activity and Cognitive Performance: A Magnetoencephalography Study , 2015 .

[121]  S. Galloway,et al.  Effects of central sympathetic inhibition on heart rate variability during steady‐state exercise in healthy humans , 2002, Clinical physiology and functional imaging.

[122]  S. Greenberg,et al.  Development and Validation of the Fatigue State Questionnaire: Preliminary Findings , 2016 .

[123]  E. Wascher,et al.  The Effects of Time on Task in Response Selection - An ERP Study of Mental Fatigue , 2015, Scientific Reports.

[124]  Robert M Califf,et al.  Biomarker definitions and their applications , 2018, Experimental biology and medicine.

[125]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[126]  Enkelejda Miho,et al.  Traditional and Digital Biomarkers: Two Worlds Apart? , 2019, Digital Biomarkers.

[127]  A. Malliani,et al.  Power spectrum analysis of heart rate variability: a tool to explore neural regulatory mechanisms. , 1994, British heart journal.

[128]  K. Mizuno,et al.  Mental fatigue caused by prolonged cognitive load associated with sympathetic hyperactivity , 2011, Behavioral and Brain Functions.

[129]  David J. Hand,et al.  Data Mining: Statistics and More? , 1998 .

[130]  L Bernardi,et al.  Low-frequency spontaneous fluctuations of R-R interval and blood pressure in conscious humans: a baroreceptor or central phenomenon? , 1994, Clinical science.

[131]  M. Ding,et al.  Change in intraindividual variability over time as a key metric for defining performance-based cognitive fatigability , 2014, Brain and Cognition.

[132]  Denver M. Y. Brown,et al.  Heart rate biofeedback attenuates effects of mental fatigue on exercise performance , 2019, Psychology of Sport and Exercise.

[133]  Pablo Laguna,et al.  Methodological Framework for Estimating the Correlation Dimension in HRV Signals , 2014, Comput. Math. Methods Medicine.

[134]  D. Bai,et al.  Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature , 2018, Psychiatry investigation.

[135]  A. Ryan,et al.  Explaining the Links between Workload, Distress, and Work-Family Conflict among School Employees: Physical, Cognitive, and Emotional Fatigue. , 2015 .

[136]  Stéphane Perrey,et al.  Quantitative Poincaré plot analysis of heart rate variability: effect of endurance training , 2003, European Journal of Applied Physiology.