Fatigue Detection Method Based on Smartphone Text Entry Performance Metrics

Workplace fatigue increases the risk of injuries or other accidents, and there is growing interest in developing means for the early detection of the signs of exhaustion. The ubiquity of smartphone use with integrated sensors allows the possibility of detecting the early signs of fatigue. Typing, chatting, Internet surfing and track-screen gestures are the most common tasks done by smartphone users in a daily manner. These tasks can be used to detect human fatigue in a non-intrusive way. This paper presents a human fatigue detection method based on virtual keyboard timing metrics which is calculated from smartphone text entry. This is achieved by using smartphone application that records keystroke events time and uses them as metrics to identify whether the user is fatigued or alert. Text entry error rate is considered as a type of psychomotor measures for fatigue/ alertness is utilized as a ground-truth metric in this study. A binary classifier based on support vector machine (SVM) classifier is suggested to identify the fatigue/alertness status of users participated in this study. The obtained results showed that the suggested method is highly accurate. The promising findings will facilitate development of a low-cost and non-intrusive mobile instrument for fatigue/alertness detection.

[1]  Andrew Beng Jin Teoh,et al.  A Survey of Keystroke Dynamics Biometrics , 2013, TheScientificWorldJournal.

[2]  Majid A. Al-Taee,et al.  Blood-glucose pattern mining algorithm for decision support in diabetes management , 2014, 2014 14th UK Workshop on Computational Intelligence (UKCI).

[3]  Qing Yang,et al.  HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users , 2015, IEEE Transactions on Information Forensics and Security.

[4]  Hanghang Tong,et al.  Activity recognition with smartphone sensors , 2014 .

[5]  Zahra J. Muhsin,et al.  Smart Bolus Estimation Taking into Account the Amount of Insulin on Board , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[6]  Valerie J. Gawron,et al.  An overview of fatigue. , 2001 .

[7]  Majid A. Al-Taee,et al.  Diagnosis and Classification of Chronic Renal Failure Utilising Intelligent Data Mining Classifiers , 2014, Int. J. Inf. Technol. Web Eng..

[8]  Fred J. Damerau,et al.  A technique for computer detection and correction of spelling errors , 1964, CACM.

[9]  Majid A. Al-Taee,et al.  HRV-based operator fatigue analysis and classification using wearable sensors , 2016, 2016 13th International Multi-Conference on Systems, Signals & Devices (SSD).

[10]  Ross Owen Phillips,et al.  What Is Fatigue and How Does It Affect the Safety Performance of Human Transport Operators? Fatigue in Transport Report I , 2014 .

[11]  Philippe A. Palanque,et al.  Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 2014, International Conference on Human Factors in Computing Systems.

[12]  Ahmed Sabbir Arif,et al.  Analysis of text entry performance metrics , 2009, 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH).

[13]  Edward D. Lemaire,et al.  Feature Selection for Wearable Smartphone-Based Human Activity Recognition with Able bodied, Elderly, and Stroke Patients , 2015, PloS one.

[14]  Ge Peng,et al.  A multimodal data set for evaluating continuous authentication performance in smartphones , 2014, SenSys.

[15]  José Neves,et al.  Monitoring Mental Fatigue through the Analysis of Keyboard and Mouse Interaction Patterns , 2013, HAIS.

[16]  I. Scott MacKenzie,et al.  Measuring errors in text entry tasks: an application of the Levenshtein string distance statistic , 2001, CHI Extended Abstracts.

[17]  I. Scott MacKenzie,et al.  Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric , 2003, CHI '03.

[18]  Zahra J. Muhsin,et al.  Mobile workflow management system based on the Internet of Things , 2016, 2016 13th International Multi-Conference on Systems, Signals & Devices (SSD).

[19]  Majid A. Al-Taee,et al.  Estimation of driver alertness using low-cost wearable devices , 2015, 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[20]  Waleed Al-Nuaimy,et al.  Prediction of driver fatigue: Approaches and open challenges , 2014, 2014 14th UK Workshop on Computational Intelligence (UKCI).

[21]  D. Dawson,et al.  Quantifying the performance impairment associated with fatigue , 1999, Journal of sleep research.