Mental Fatigue Estimation Based on Facial Expression Change during Speech

In this study, we propose the method for estimating the user’s fatigue due to mental load like a desk work based on information of facial expressions. It is, however, difficult to estimate the mental fatigue using ambient sensors information because of a small change of user’s appearance. Thus the user’s mental fatigue is measured based on his/her facial expression during speech. As experimental results, recognition rate base on facial expression during speech marks higher rate (at most 89%) than that based on facial expression without speech. In addition, we conduct the another experiment for different types of workload, it is confirmed that proposed method is effective for various fatigue.

[1]  Chong Zhang,et al.  Automatic recognition of cognitive fatigue from physiological indices by using wavelet packet transform and kernel learning algorithms , 2009, Expert Syst. Appl..

[2]  Eric Friets,et al.  Fatigue estimation using voice analysis , 2007, Behavior research methods.

[3]  Y Nomura,et al.  [Characteristics of the motor units in the orbicularis oris muscle]. , 1986, Kokubyo Gakkai zasshi. The Journal of the Stomatological Society, Japan.

[4]  Kun Li,et al.  EEG-based detection and evaluation of fatigue caused by watching 3DTV , 2013, Displays.

[5]  Shin-ichi Takeda,et al.  Evaluation of Feeling of Fatigue Using by Analysis of Facial Expression , 2004 .

[6]  Chongxun Zheng,et al.  Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. , 2012, Accident; analysis and prevention.

[7]  Shinji Miyake,et al.  Subjective mental workload assessment technique , 1993 .

[8]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[9]  Jianping Liu,et al.  EEG-based estimation of mental fatigue by using KPCA-HMM and complexity parameters , 2010, Biomed. Signal Process. Control..

[10]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[11]  Tianwei Shi,et al.  Real-Time EEG-Based Detection of Fatigue Driving Danger for Accident Prediction , 2015, Int. J. Neural Syst..

[12]  Michel Besserve,et al.  Multimodal information improves the rapid detection of mental fatigue , 2013, Biomed. Signal Process. Control..

[13]  S. Kurahashi,et al.  Development of the Uchida-Kraepelin psychodiagnostic test in Japan , 1957 .

[14]  Rongrong Fu,et al.  Automated Detection of Driver Fatigue Based on Entropy and Complexity Measures , 2014, IEEE Transactions on Intelligent Transportation Systems.