Automated Measurement of Head Movement Synchrony during Dyadic Depression Severity Interviews

With few exceptions, most research in automated assessment of depression has considered only the patient’s behavior to the exclusion of the therapist’s behavior. We investigated the interpersonal coordination (synchrony) of head movement during patient-therapist clinical interviews. Our sample consisted of patients diagnosed with major depressive disorder. They were recorded in clinical interviews (Hamilton Rating Scale for Depression, HRSD) at 7-week intervals over a period of 21 weeks. For each session, patient and therapist 3D head movement was tracked from 2D videos. Head angles in the horizontal (pitch) and vertical (yaw) axes were used to measure head movement. Interpersonal coordination of head movement between patients and therapists was measured using windowed cross-correlation. Patterns of coordination in head movement were investigated using the peak picking algorithm. Changes in head movement coordination over the course of treatment were measured using a hierarchical linear model (HLM). The results indicated a strong effect for patient-therapist head movement synchrony. Within-dyad variability in head movement coordination was found to be higher than between-dyad variability, meaning that differences over time in a dyad were higher as compared to the differences between dyads. Head movement synchrony did not change over the course of treatment with change in depression severity. To the best of our knowledge, this study is the first attempt to analyze the mutual influence of patient-therapist head movement in relation to depression severity.

[1]  J. Cohn,et al.  Dyadic Behavior Analysis in Depression Severity Assessment Interviews , 2014, ICMI.

[2]  Mohammad H. Mahoor,et al.  Social risk and depression: Evidence from manual and automatic facial expression analysis , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[3]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  N. Ambady,et al.  Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. , 1993 .

[5]  Kacem Anis,et al.  Detecting Depression Severity by Interpretable Representations of Motion Dynamics , 2018 .

[6]  D. Bates,et al.  Newton-Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measures Data , 1988 .

[7]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[8]  Fabien Ringeval,et al.  AVEC 2016: Depression, Mood, and Emotion Recognition Workshop and Challenge , 2016, AVEC@ACM Multimedia.

[9]  M. Caligiuri,et al.  Motor and cognitive aspects of motor retardation in depression. , 2000, Journal of affective disorders.

[10]  Michael Wagner,et al.  Characterising depressed speech for classification , 2013, INTERSPEECH.

[11]  Laszlo A. Jeni,et al.  Spontaneous facial expression in unscripted social interactions can be measured automatically , 2015, Behavior research methods.

[12]  H. Sackeim,et al.  Psychomotor symptoms of depression. , 1997, The American journal of psychiatry.

[13]  Jeffrey F. Cohn,et al.  Intra- and Interpersonal Functions of Head Motion in Emotion Communication , 2014, RFMIR '14.

[14]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[15]  Fernando De la Torre,et al.  Detecting depression from facial actions and vocal prosody , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[16]  Albert A. Rizzo,et al.  Automatic audiovisual behavior descriptors for psychological disorder analysis , 2014, Image Vis. Comput..

[17]  G. Arbanas.  Diagnostic and Statistical Manual of Mental Disorders (DSM-5) , 2015 .

[18]  G. Laurent,et al.  Temporal Representations of Odors in an Olfactory Network , 1996, The Journal of Neuroscience.

[19]  Mohammad H. Mahoor,et al.  Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses , 2014, Image Vis. Comput..

[20]  A. J. Fridlund.  Sociality of Solitary Smiling: Potentiation by an Implicit Audience , 1991 .

[21]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[22]  Jeffrey M Girard,et al.  Automated Audiovisual Depression Analysis. , 2015, Current opinion in psychology.

[23]  J. Gottman,et al.  The Analysis of Dominance and Bidirectionality in Social Development. , 1981 .

[24]  R. Hu.  Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) , 2003 .

[25]  M. Kret,et al.  Connecting minds and sharing emotions through mimicry: A neurocognitive model of emotional contagion , 2017, Neuroscience & Biobehavioral Reviews.

[26]  M Vannicelli,et al.  Speaking to and about patients: predicting therapists' tone of voice. , 1984, Journal of consulting and clinical psychology.

[27]  Jeffrey F. Cohn,et al.  Detecting Depression Severity from Vocal Prosody , 2013, IEEE Transactions on Affective Computing.

[28]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[29]  Jeffrey F. Cohn,et al.  Dynamic Multimodal Measurement of Depression Severity Using Deep Autoencoding , 2018, IEEE Journal of Biomedical and Health Informatics.

[30]  Takeo Kanade,et al.  Dense 3D face alignment from 2D videos in real-time , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[31]  Jeffrey F. Cohn.  Beyond group differences: specificity of nonverbal behavior and interpersonal communication to depression severity , 2013, AVEC@ACM Multimedia.

[32]  George D. C. Cavalcanti,et al.  Enhanced real-time head pose estimation system for mobile device , 2014, Integr. Comput. Aided Eng..

[33]  Jeffrey F. Cohn,et al.  Interpersonal Coordination of HeadMotion in Distressed Couples , 2014, IEEE Transactions on Affective Computing.

[34]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[35]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[36]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[37]  R. DeRubeis,et al.  Antidepressant drug effects and depression severity: a patient-level meta-analysis. , 2010, JAMA.

[38]  S. Boker,et al.  Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. , 2002, Psychological methods.

[39]  M. Hamilton,et al.  Development of a rating scale for primary depressive illness. , 1967, The British journal of social and clinical psychology.

[40]  Roland Göcke,et al.  Modeling spectral variability for the classification of depressed speech , 2013, INTERSPEECH.

[41]  Albrecht Rüdiger,et al.  Spectrum and spectral density estimation by the Discrete Fourier transform (DFT), including a comprehensive list of window functions and some new at-top windows , 2002 .

[42]  Roland Göcke,et al.  Relative Body Parts Movement for Automatic Depression Analysis , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[43]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[44]  J. Cohn,et al.  Automated Measurement of Facial Expression in Infant-Mother Interaction: A Pilot Study. , 2009, Infancy : the official journal of the International Society on Infant Studies.