Modeling Stress Using Thermal Facial Patterns: A Spatio-temporal Approach

Stress is a serious concern facing our world today, motivating the development of better objective understanding using non-intrusive means for stress recognition. The aim for the work was to use thermal imaging of facial regions to detect stress automatically. The work uses facial regions captured in videos in thermal (TS) and visible (VS) spectrums and introduces our database ANU StressDB. It describes the experiment conducted for acquiring TS and VS videos of observers of stressed and not-stressed films for the ANU StressDB. Further, it presents an application of local binary patterns on three orthogonal planes (LBP-TOP) on VS and TS videos for stress recognition. It proposes a novel method to capture dynamic thermal patterns in histograms (HDTP) to utilize thermal and spatio-temporal characteristics associated in TS videos. Individual-independent support vector machine classifiers were developed for stress recognition. Results show that a fusion of facial patterns from VS and TS videos produced significantly better stress recognition rates than patterns from only VS or TS videos with p <; 0.01. The best stress recognition rate was 72% and it was obtained from HDTP features fused with LBP-TOP features for TS and VS videos respectively.

[1]  A Sali,et al.  Mind‐Body Medicine: Stress and Its Impact on Overall Health and Longevity , 2005, Annals of the New York Academy of Sciences.

[2]  Mark A. Richardson,et al.  Emotional & physical stress detection and classification using thermal imaging technique , 2009, ICDP.

[3]  L Hoffman-Goetz,et al.  Exercise and the immune system: a model of the stress response? , 1994, Immunology today.

[4]  C. Holahan,et al.  Life stressors, resistance factors, and improved psychological functioning: an extension of the stress resistance paradigm. , 1990, Journal of personality and social psychology.

[5]  Tamás D. Gedeon,et al.  Objective measures, sensors and computational techniques for stress recognition and classification: A survey , 2012, Comput. Methods Programs Biomed..

[6]  Leonardo Trujillo,et al.  Visual learning of texture descriptors for facial expression recognition in thermal imagery , 2007, Comput. Vis. Image Underst..

[7]  Roland Göcke,et al.  Neural-net classification for spatio-temporal descriptor based depression analysis , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[8]  William R. Lovallo,et al.  Stress and Health: Biological and Psychological Interactions , 1997 .

[9]  Vitomir Struc,et al.  Gabor-Based Kernel Partial-Least-Squares Discrimination Features for Face Recognition , 2009, Informatica.

[10]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  David Sander,et al.  Thermal Analysis of Facial Muscles Contractions , 2011, IEEE Transactions on Affective Computing.

[13]  Marian Stewart Bartlett,et al.  Weakly supervised pain localization using multiple instance learning , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[14]  Takafumi Ushiyama,et al.  Analysis of heart rate variability as an index of noncardiac surgical stress , 2008, Heart and Vessels.

[15]  Tao Lin,et al.  Do physiological data relate to traditional usability indexes? , 2005, OZCHI.

[16]  L. J. M. Rothkrantz,et al.  DETECTING STRESS USING EYE BLINKS AND BRAIN ACTIVITY FROM EEG SIGNALS , 2009 .

[17]  Leonardo Trujillo,et al.  Automatic Feature Localization in Thermal Images for Facial Expression Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[18]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[19]  Stefan Rose-John,et al.  Virokines in the Pathogenesis of Cancer: Focus on Human Herpesvirus 8 , 2004, Annals of the New York Academy of Sciences.

[20]  Zhiwei Zhu,et al.  A Real-Time Human Stress Monitoring System Using Dynamic Bayesian Network , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[21]  Vitomir Struc,et al.  The Complete Gabor-Fisher Classifier for Robust Face Recognition , 2010, EURASIP J. Adv. Signal Process..

[22]  Matti Pietikäinen,et al.  Facial expression recognition from near-infrared video sequences , 2008, 2008 19th International Conference on Pattern Recognition.

[23]  Dimitris N. Metaxas,et al.  Optical computer recognition of facial expressions associated with stress induced by performance demands. , 2005, Aviation, space, and environmental medicine.

[24]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[25]  John Pluta,et al.  Gender difference in neural response to psychological stress. , 2007, Social cognitive and affective neuroscience.

[26]  C. Kirschbaum,et al.  Chronic psychological stress and the regulation of pro-inflammatory cytokines: a glucocorticoid-resistance model. , 2002, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[27]  Tamás D. Gedeon,et al.  Stress Classification for Gender Bias in Reading , 2011, ICONIP.

[28]  Tamás D. Gedeon,et al.  Emotion recognition using PHOG and LPQ features , 2011, Face and Gesture 2011.

[29]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[30]  Armando Barreto,et al.  Stress Recognition Using Non-invasive Technology , 2006, FLAIRS.

[31]  L. R. John,et al.  Quantifying Mental Relaxation with EEG for use in Computer Games , 2006, International Conference on Internet Computing.

[32]  Fang Chen,et al.  Galvanic skin response (GSR) as an index of cognitive load , 2007, CHI Extended Abstracts.

[33]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.