An approach for automatically measuring facial activity in depressed subjects

This paper is motivated by Ellgring's work in non-verbal communication in depression to measure and compare the levels of facial activity, before and after treatment, of endogenous and neurotic depressives. Similar to that work, we loosely associate the measurements with action units (AU) groups from the facial action coding system (FACS). However, we use the neologism region units (RU) to describe regions of the face that encapsulate AUs. In contrast to Ell-gring's approach, we automatically generate the measurements and provide both prototypical expression recognition and RU-specific activity measurements. Latency between expressions is also measured and the system is conducive to comparison across groups and individual subjects. By using active appearance models (AAM) to locate the fiduciary facial points, and multiboost to classify prototypical expressions and the RUs, we can provide a simple, objective, flexible and cost-effective means of automatically measuring facial activity.

[1]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[2]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[3]  R. Gur,et al.  Automated video-based facial expression analysis of neuropsychiatric disorders , 2008, Journal of Neuroscience Methods.

[4]  R. de Raedt,et al.  Deficient inhibition of emotional information in depression. , 2006, Journal of affective disorders.

[5]  I. Gotlib,et al.  Is this happiness I see? Biases in the identification of emotional facial expressions in depression and social phobia. , 2006, Journal of abnormal psychology.

[6]  Jerome M. Schneck,et al.  The Phenomena of Depressions , 1962 .

[7]  E. Bullmore,et al.  Emotional memory: Separating content and context , 2005, Psychiatry Research: Neuroimaging.

[8]  Anupinder Kaur,et al.  Prevalence of Veraguth's Eyelid Folds During Depression in Different Ethnic Groups , 2004, The Journal of nervous and mental disease.

[9]  J. Harrigan,et al.  How do you look when feeling anxious? Facial displays of anxiety , 1996 .

[10]  T. Dalgleish,et al.  Handbook of cognition and emotion , 1999 .

[11]  H. Ellgring.  Nonverbal communication in depression , 1989 .

[12]  Eva H. Telzer,et al.  Amygdala and nucleus accumbens activation to emotional facial expressions in children and adolescents at risk for major depression. , 2008, The American journal of psychiatry.

[13]  Karsten Berns,et al.  Visual-Based Emotion Detection for Natural Man-Machine Interaction , 2008, KI.

[14]  Robin M. Murray,et al.  Neural Responses to Sad Facial Expressions in Major Depression Following Cognitive Behavioral Therapy , 2008, Biological Psychiatry.

[15]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[16]  R. Rosenthal,et al.  Detecting State and Trait Anxiety from Auditory and Visual Cues: A Meta-Analysis , 2004, Personality & social psychology bulletin.

[17]  R. Gibson,et al.  What the Face Reveals , 2002 .

[18]  P. Ekman,et al.  Behavioral markers and recognizability of the smile of enjoyment. , 1993, Journal of personality and social psychology.

[19]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[20]  P. Ekman.  Unmasking The Face , 1975 .

[21]  J. Cohn,et al.  Impact of depression on response to comedy: a dynamic facial coding analysis. , 2007, Journal of abnormal psychology.

[22]  Byung-Joo Ham,et al.  The neural substrates of affective processing toward positive and negative affective pictures in patients with major depressive disorder , 2007, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[23]  P. Ekman.  Darwin, Deception, and Facial Expression , 2003, Annals of the New York Academy of Sciences.

[24]  J. Harrigan,et al.  Fooled by a Smile: Detecting Anxiety in Others , 1997 .

[25]  B. Renneberg,et al.  Facial expression of emotions in borderline personality disorder and depression. , 2005, Journal of behavior therapy and experimental psychiatry.

[26]  F. Wallhoff,et al.  The Facial Expressions and Emotions Database Homepage (FEEDTUM) , 2005 .

[27]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[28]  Nick Medford,et al.  Emotional memory for words: Separating content and context , 2007 .

[29]  I. Gotlib,et al.  Selective attention to emotional faces following recovery from depression. , 2007, Journal of abnormal psychology.

[30]  P. Ekman.  Emotion in the human face , 1982 .

[31]  Karin Mogg,et al.  Attentional Bias in Generalized Anxiety Disorder Versus Depressive Disorder , 2005, Cognitive Therapy and Research.

[32]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.