Heart rate estimation from facial videos for depression analysis

Automated facial video analysis is useful in numerous health care applications. For example, spatio-temporal analysis of such videos has been previously done for assisting clinicians in the diagnosis of depression. Physiological measures, such as an individual's heart rate, provide very important cues to understand a person's mental health. Unobtrusively estimated heart rate has not been previously used to analyse individuals' mental health. In this paper, we automatically estimate heart rate activity from facial videos. We then study the association of the estimated heart rate activity with the person's mental health, as diagnosed by clinicians. Specifically, from the heart rate activity in response to watching different movies, we classify individuals as either depressed or healthy. The efficacy of the proposed scheme is demonstrated by experimental evaluations on a clinically validated dataset. Our results suggest unobtrusively estimated heart rate to be very effective for depression analysis.

[1]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[2]  Eliathamby Ambikairajah,et al.  Spectro-temporal analysis of speech affected by depression and psychomotor retardation , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Mohammed Bennamoun,et al.  An Automatic Framework for Textured 3D Video-Based Facial Expression Recognition , 2014, IEEE Transactions on Affective Computing.

[5]  Mohammed Bennamoun,et al.  Evaluation of Spatiotemporal Detectors and Descriptors for Facial Expression Recognition , 2012, 2012 5th International Conference on Human System Interactions.

[6]  Michael Wagner,et al.  Exploring Eye Activity as an Indication of Emotional States Using an Eye-Tracking Sensor , 2014 .

[7]  Junzhou Huang,et al.  Pose-Free Facial Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Shape Model , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Yen-Ting Chen,et al.  Physiological signal analysis for patients with depression , 2011, 2011 4th International Conference on Biomedical Engineering and Informatics (BMEI).

[9]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[10]  Olga V. Demler,et al.  The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). , 2003, JAMA.

[11]  Spielberg Te,et al.  Treating depression and anxiety in primary care. , 1992 .

[12]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[13]  V. Manicavasagar,et al.  A review of depression diagnosis and management , 2012 .

[14]  Roland Göcke,et al.  An approach for automatically measuring facial activity in depressed subjects , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[15]  W. Marsden.  I and J , 2012 .

[16]  Szymon Fedor.  Can We Predict Depression From the Asymmetry of Electrodermal Activity , 2016 .