A data analytic framework for physical fatigue management using wearable sensors

Abstract The use of expert systems in optimizing and transforming human performance has been limited in practice due to the lack of understanding of how an individual’s performance deteriorates with fatigue accumulation, which can vary based on both the worker and the workplace conditions. As a first step toward realizing the human-centered approach to artificial intelligence and expert systems, this paper lays the foundation for a data analytic approach to managing fatigue in physically-demanding workplaces. The proposed framework capitalizes on continuously collected human performance data from wearable sensor technologies, and is centered around four distinct phases of fatigue: (a) detection, where machine learning methodologies are deployed to detect the occurrence of fatigue; (b) identification, where key features relating to the fatigue occurrence is to be identified; (c) diagnosis, where the fatigue mode is identified based on the knowledge generated in the previous two phases; and (d) recovery, where a suitable intervention is applied to return the worker to mitigate the detrimental effects of fatigue on the worker. Moreover, the framework establishes criteria for feature and machine learning algorithm selection for fatigue management. Two specific application cases of the framework, for two types of manufacturing-related tasks, are presented. Based on the proposed framework and a large number of test sets used in the two case studies, we have shown that: (i) only one wearable sensor is needed for fatigue detection with an average accuracy of  ≥ 0.850 and a random forest model comprised of

[1]  Cheryl L. Marcham,et al.  Online Vs. Conventional: Safety Training Approaches , 2019 .

[2]  Mohammad T. Khasawneh,et al.  Simulation optimization approach for patient scheduling at destination medical centers , 2020, Expert Syst. Appl..

[3]  Fadel M. Megahed,et al.  Monitoring worker fatigue using wearable devices: A case study to detect changes in gait parameters , 2019, Journal of Quality Technology.

[4]  Mirko Perkusich,et al.  Early diagnosis of gastrointestinal cancer by using case-based and rule-based reasoning , 2016, Expert Syst. Appl..

[5]  Marcus Yung,et al.  Detecting within- and between-day manifestations of neuromuscular fatigue at work: an exploratory study , 2014, Ergonomics.

[6]  Cristiano André da Costa,et al.  Survey of conversational agents in health , 2019, Expert Syst. Appl..

[7]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[8]  Wan Chul Yoon,et al.  Detecting user attention to video segments using interval EEG features , 2019, Expert Syst. Appl..

[9]  Ranjana K. Mehta,et al.  The effects of obesity, age, and relative workload levels on handgrip endurance. , 2015, Applied ergonomics.

[10]  Thurmon E. Lockhart,et al.  Gait analysis to classify external load conditions using linear discriminant analysis. , 2009, Human movement science.

[11]  John L. Crassidis,et al.  Hip and Trunk Kinematics Estimation in Gait Through Kalman Filter Using IMU Data at the Ankle , 2018, IEEE Sensors Journal.

[12]  K. Matsuoka,et al.  Effect of prolonged free-walking fatigue on gait and physiological rhythm. , 2004, Journal of biomechanics.

[13]  Paolo Bonato,et al.  Reliability of EMG time-frequency measures of fatigue during repetitive lifting. , 2002, Medicine and science in sports and exercise.

[14]  Roberto Lamberts,et al.  Technological innovations to assess and include the human dimension in the building-performance loop: A review , 2019, Energy and Buildings.

[15]  Deok-Hwan Kim,et al.  Real-time gait subphase detection using an EMG signal graph matching (ESGM) algorithm based on EMG signals , 2017, Expert Syst. Appl..

[16]  O. Kajimoto,et al.  Development of a Method of Evaluation of Fatigue and Its Economic Impacts , 2008 .

[17]  Julio López,et al.  Simultaneous feature selection and heterogeneity control for SVM classification: An application to mental workload assessment , 2020, Expert Syst. Appl..

[18]  Anatol G. Feldman,et al.  Movement reorganization to compensate for fatigue during sawing , 2002, Experimental Brain Research.

[19]  J. Evans,et al.  Postural and trunk muscle response to sudden release during stoop lifting tasks before and after fatigue of the trunk erector muscles , 2004, Ergonomics.

[20]  Anil Mital,et al.  A guide to manual materials handling , 1993 .

[21]  Gerhard Tröster,et al.  Monitoring Kinematic Changes With Fatigue in Running Using Body-Worn Sensors , 2012, IEEE Transactions on Information Technology in Biomedicine.

[22]  Lidong Wang Machine Learning in Big Data , 2015 .

[23]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[24]  Tatsuo Nakajima,et al.  Feature Selection and Activity Recognition from Wearable Sensors , 2006, UCS.

[25]  L Quagliarella,et al.  An interactive fall and loss of consciousness detector system. , 2008, Gait & posture.

[26]  Sang Won Yoon,et al.  A dynamic gradient boosting machine using genetic optimizer for practical breast cancer prognosis , 2019, Expert Syst. Appl..

[27]  Marc Loriol,et al.  A sociological stance on fatigue and tiredness: Social inequalities, norms and representations , 2017, Neurophysiologie Clinique/Clinical Neurophysiology.

[28]  Min Zhao,et al.  Multivariate autoregressive models and kernel learning algorithms for classifying driving mental fatigue based on electroencephalographic , 2011, Expert Syst. Appl..

[29]  L. R. Altimari,et al.  The relationship between rating of perceived exertion and muscle activity during exhaustive constant-load cycling. , 2010, International journal of sports medicine.

[30]  D. Gagnon,et al.  The comparison of trunk muscles EMG activation between subjects with and without chronic low back pain during flexion-extension and lateral bending tasks. , 2000, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[31]  Pietro Siciliano,et al.  Supervised machine learning scheme for electromyography-based pre-fall detection system , 2018, Expert Syst. Appl..

[32]  Serhan Ziya,et al.  Priority Assignment Under Imperfect Information on Customer Type Identities , 2009, Manuf. Serv. Oper. Manag..

[33]  Damien Trentesaux,et al.  Towards human-based industrial cyber-physical systems , 2018, 2018 IEEE Industrial Cyber-Physical Systems (ICPS).

[34]  J. Ricci,et al.  Fatigue in the U.S. Workforce: Prevalence and Implications for Lost Productive Work Time , 2007, Journal of occupational and environmental medicine.

[35]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[36]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[37]  Murat Kulahci,et al.  Predictive Manufacturing: A Classification Strategy to Predict Product Failures , 2020 .

[38]  Jorge Pinho de Sousa,et al.  A two-phase MILP approach to integrate order, customer and manufacturer characteristics into Dynamic Manufacturing Network formation and operational planning , 2018, Expert Syst. Appl..

[39]  Massimiliano de Leoni,et al.  Aligning Real Process Executions and Prescriptive Process Models through Automated Planning , 2017, Expert Syst. Appl..

[40]  Ridha Soua,et al.  Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges , 2017, IEEE Transactions on Vehicular Technology.

[41]  Jian Zhang,et al.  Classifying Lower Extremity Muscle Fatigue During Walking Using Machine Learning and Inertial Sensors , 2013, Annals of Biomedical Engineering.

[42]  Lin Lu,et al.  Effects of Task Type, Task Duration, and Age on Body Kinematics and Subjective Fatigue , 2017 .

[43]  Alexandros Stamatakis,et al.  How Many Bootstrap Replicates Are Necessary? , 2009, RECOMB.

[44]  Lin Lu,et al.  Predicting short-term stock prices using ensemble methods and online data sources , 2018, Expert Syst. Appl..

[45]  Christoph H. Glock,et al.  Human factors in order picking: a content analysis of the literature , 2017, Int. J. Prod. Res..

[46]  Giorgio Mossa,et al.  Productivity and ergonomic risk in human based production systems: A job-rotation scheduling model , 2016 .

[47]  Fadel M. Megahed,et al.  Monitoring and Change Point Estimation of Normal (In-Control) and Fatigued (Out-of-Control) State in Workers , 2016 .

[48]  Fabio Massimo Zanzotto Human-in-the-loop Artificial Intelligence , 2017, ArXiv.

[49]  Henri Pierreval,et al.  A simulation-optimization based heuristic for the online assignment of multi-skilled workers subjected to fatigue in manufacturing systems , 2017, Comput. Ind. Eng..

[50]  Sylvie Charbonnier,et al.  EEG index for control operators' mental fatigue monitoring using interactions between brain regions , 2016, Expert Syst. Appl..

[51]  R. Koster,et al.  Safety Does Not Happen by Accident: Antecedents To A Safer Warehouse , 2016 .

[52]  Ali Dag,et al.  A probabilistic data-driven framework for scoring the preoperative recipient-donor heart transplant survival , 2016, Decis. Support Syst..

[53]  Domagoj Jakobovic,et al.  A survey of dispatching rules for the dynamic unrelated machines environment , 2018, Expert Syst. Appl..

[54]  P. Dolan,et al.  Repetitive lifting tasks fatigue the back muscles and increase the bending moment acting on the lumbar spine. , 1998, Journal of biomechanics.

[55]  Jack T Dennerlein,et al.  Wrist and shoulder posture and muscle activity during touch-screen tablet use: effects of usage configuration, tablet type, and interacting hand. , 2013, Work.

[56]  Fredrik Ødegaard,et al.  Measuring the Contribution of Workers' Health and Psychosocial Work‐Environment on Production Efficiency , 2014 .

[57]  Ataollah Abbasi,et al.  A new algorithm for kinematic analysis of handwriting data; towards a reliable handwriting-based tool for early detection of alzheimer's disease , 2018, Expert Syst. Appl..

[58]  Zahra Sedighi Maman,et al.  A data-driven approach to modeling physical fatigue in the workplace using wearable sensors. , 2017, Applied ergonomics.

[59]  Muhammad Hisyam Lee,et al.  Penalized logistic regression with the adaptive LASSO for gene selection in high-dimensional cancer classification , 2015, Expert Syst. Appl..

[60]  M. A. Nussbaum,et al.  Effects of lumbar extensor fatigue and fatigue rate on postural sway , 2004, European Journal of Applied Physiology.

[61]  Laura A. Wojcik,et al.  Age and gender moderate the effects of localized muscle fatigue on lower extremity joint torques used during quiet stance. , 2011, Human movement science.

[62]  U Della Croce,et al.  Muscle Fatigue and Fatigue-Related Biomechanical Changes During a Cyclic Lifting Task , 2003, Spine.

[63]  Xiaoping Yun,et al.  Self-contained Position Tracking of Human Movement Using Small Inertial/Magnetic Sensor Modules , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[64]  Xiang T. R. Kong,et al.  Industrial wearable system: the human-centric empowering technology in Industry 4.0 , 2018, Journal of Intelligent Manufacturing.

[65]  R. Moe-Nilssen,et al.  Physical fatigue affects gait characteristics in older persons. , 2007, The journals of gerontology. Series A, Biological sciences and medical sciences.

[66]  Songfa Zhong,et al.  Recover Overnight? Work Interruption and Worker Productivity , 2016 .

[67]  Steven Morrison,et al.  Lumbar and cervical erector spinae fatigue elicit compensatory postural responses to assist in maintaining head stability during walking. , 2006, Journal of applied physiology.

[68]  Gary King,et al.  Logistic Regression in Rare Events Data , 2001, Political Analysis.

[69]  Paulo Cortez,et al.  Automatic visual detection of human behavior: A review from 2000 to 2014 , 2015, Expert Syst. Appl..

[70]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[71]  Andrina Granic,et al.  Technology acceptance model: a literature review from 1986 to 2013 , 2014, Universal Access in the Information Society.

[72]  Ian Hickie,et al.  The Chronic Fatigue Syndrome: A Comprehensive Approach to Its Definition and Study , 1994, Annals of Internal Medicine.

[73]  Richard F. Sesek,et al.  Barriers to the Adoption of Wearable Sensors in the Workplace: A Survey of Occupational Safety and Health Professionals , 2018, Hum. Factors.

[74]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[75]  Juan Luis Castro,et al.  A multi-agent conversational system with heterogeneous data sources access , 2016, Expert Syst. Appl..

[76]  Michelle Karg,et al.  Human Movement Analysis as a Measure for Fatigue: A Hidden Markov-Based Approach , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[77]  Arnulfo Aurelio Naranjo-Flores,et al.  Human Factors and Ergonomics for Lean Manufacturing Applications , 2014 .

[78]  Gerhard Tröster,et al.  SwimMaster: a wearable assistant for swimmer , 2009, UbiComp.

[79]  C. Foster,et al.  Monitoring training in athletes with reference to overtraining syndrome. , 1998, Medicine and science in sports and exercise.

[80]  Kolja Kuhnlenz,et al.  Expression and Automatic Recognition of Exhaustion in Natural Walking , 2008 .

[81]  Fadel M. Megahed,et al.  Stock market one-day ahead movement prediction using disparate data sources , 2017, Expert Syst. Appl..

[82]  Alec R. Levenson WORKPLACE FATIGUE IS A SYSTEMS PROBLEM , 2017 .

[83]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[84]  J. Willson,et al.  Plantar loading and cadence alterations with fatigue. , 1999, Medicine and science in sports and exercise.

[85]  Steven Truijen,et al.  Visual deprivation leads to gait adaptations that are age- and context-specific: II. Kinematic parameters. , 2009, Gait & posture.

[86]  Angelica E. Lang,et al.  The combined fatigue effects of sequential exposure to seated whole body vibration and physical, mental, or concurrent work demands , 2017, PloS one.

[87]  Ehsan T. Esfahani,et al.  A machine learning approach to detect changes in gait parameters following a fatiguing occupational task , 2018, Ergonomics.

[88]  Fadel M. Megahed,et al.  Interventions to Mitigate Fatigue Induced by Physical Work: A Systematic Review of Research Quality and Levels of Evidence for Intervention Efficacy , 2021, Hum. Factors.

[89]  R J Shephard,et al.  Revision of the Physical Activity Readiness Questionnaire (PAR-Q). , 1992, Canadian journal of sport sciences = Journal canadien des sciences du sport.

[90]  Loo Hay Lee,et al.  Tree based searching approaches for integrated vehicle dispatching and container allocation in a transshipment hub , 2017, Expert Syst. Appl..

[91]  Theodore F. Towse,et al.  Human skeletal muscle responses vary with age and gender during fatigue due to incremental isometric exercise. , 2002, Journal of applied physiology.

[92]  Lin Lu,et al.  A survey of the prevalence of fatigue, its precursors and individual coping mechanisms among U.S. manufacturing workers. , 2017, Applied ergonomics.

[93]  S. Ramakrishnan,et al.  Extraction and analysis of multiple time window features associated with muscle fatigue conditions using sEMG signals , 2014, Expert Syst. Appl..

[94]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[95]  Eric W.T. Ngai,et al.  Decision-making techniques in supplier selection: Recent accomplishments and what lies ahead , 2020, Expert Syst. Appl..

[96]  Saeid Nahavandi,et al.  A Review on Ocular Biomechanic Models for Assessing Visual Fatigue in Virtual Reality , 2018, IEEE Access.

[97]  Davide Anguita,et al.  Emerging trends in machine learning: beyond conventional methods and data , 2018, ESANN.

[98]  E. Demerouti,et al.  Decision-Making Processes in the Workplace: How Exhaustion, Lack of Resources and Job Demands Impair Them and Affect Performance , 2017, Front. Psychol..