Play with me — Measuring a child's engagement in a social interaction

Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child's engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviour modelling and can be a good substitute in the absence of accurate high-level features.