Head Nod Detection in Dyadic Conversations

In face-to-face interactions, head gestures play an important role as one of the back-channel signals. As one of them, head nods can be used to display the approval or interest of listeners as a feedback in dyadic conversations. Hence detection of head nods is expected to improve understanding of the given feedback and to improve human-computer interaction. This study targets to detect head nods in the purpose of making human-computer interaction more human like. In the process, 3D head model is obtained by the Microsoft Kinect and the Openface application. Binary classification is performed on spectral features, which are extracted from 3D head motion, with the Support Vector Machine (SVM) classifier. Consequently, upon the classification, ‘head nod’ or ‘not head nod’ outputs are obtained. In the experimental studies, head nod detection accuracy is obtained as 92% for Microsoft Kinect and 91% for Openface over the Joker dataset.

[1]  Gang Rong,et al.  A real-time head nod and shake detector using HMMs , 2003, Expert Syst. Appl..

[2]  Guillaume Dubuisson Duplessis,et al.  Multifaceted Engagement in Social Interaction with a Machine: The JOKER Project , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[3]  Trevor Darrell,et al.  Contextual recognition of head gestures , 2005, ICMI '05.

[4]  Jean-Marc Odobez,et al.  Head Nod Detection from a Full 3D Model , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Ashish Kapoor,et al.  A real-time head nod and shake detector , 2001, PUI '01.

[7]  David S. Monaghan,et al.  Real-time head nod and shake detection for continuous human affect recognition , 2013, 2013 14th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS).

[8]  Louis-Philippe Morency,et al.  OpenFace 2.0: Facial Behavior Analysis Toolkit , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[9]  U. Hadar,et al.  Head movement during listening turns in conversation , 1985 .

[10]  Jean-Marc Odobez,et al.  Using self-context for multimodal detection of head nods in face-to-face interactions , 2012, ICMI '12.