Video analysis of approach-avoidance behaviors of teenagers speaking with virtual agents

Analysis of non-verbal behaviors in HCI allows understanding how individuals apprehend and adapt to different situations of interaction. This seems particularly relevant when considering tasks such as speaking in a foreign language, which is known to elicit anxiety. This is even truer for young users for whom negative pedagogical feedbacks might have a strong negative impact on their motivation to learn. In this paper, we consider the approach-avoidance behaviors of teenagers speaking with virtual agents when using an e-learning platform for learning English. We designed an algorithm for processing the video of these teenagers outside laboratory conditions (e.g. a classical collective classroom in a secondary school) using a webcam. This algorithm processes the video of the user and computes the inter-ocular distance. The anxiety of the users is also collected with questionnaires. Results show that the inter-ocular distance enables to discriminate between approach and avoidance behaviors of teenagers reacting to positive or negative stimulus. This simple metric collected via video processing enables to detect an approach behavior related to a positive stimulus and an avoidance behavior related to a negative stimulus. Furthermore, we observed that these automatically detected approach-avoidance behaviors are correlated with anxiety.

[1]  Athanasios Katsamanis,et al.  Estimation of ordinal approach-avoidance labels in dyadic interactions: Ordinal logistic regression approach , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Rolf A. Zwaan,et al.  Posture as Index for Approach-Avoidance Behavior , 2012, PloS one.

[3]  M. Thüring,et al.  Usability, aesthetics and emotions in human–technology interaction , 2007 .

[4]  Shinjiro Kawato,et al.  Real-time detection of nodding and head-shaking by directly detecting and tracking the "between-eyes" , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[5]  Rolf A. Zwaan,et al.  Approach and avoidance as action effects , 2008, Quarterly journal of experimental psychology.

[6]  N. Liberman,et al.  Approach and avoidance strength during goal attainment: regulatory focus and the "goal looms larger" effect. , 1998, Journal of personality and social psychology.

[7]  Lj Stuart,et al.  International Journal of Computer Application , 2014 .

[8]  Mido Chang,et al.  Engagement states and learning from educational games. , 2013, New directions for child and adolescent development.

[9]  Christopher E. Peters,et al.  A head movement propensity model for animating gaze shifts and blinks of virtual characters , 2010, Comput. Graph..

[10]  T. Chartrand,et al.  THE UNBEARABLE AUTOMATICITY OF BEING , 1999 .

[11]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[12]  R. Rapee,et al.  A cognitive-behavioral model of anxiety in social phobia. , 1997, Behaviour research and therapy.

[13]  E. Skinner,et al.  Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. , 1993 .

[14]  Rishi Pal Singh,et al.  Automatic Speech Recognition: A Review , 2012 .

[15]  J. Gray,et al.  The neuropsychology of emotion and personality. , 1987 .

[16]  Zhiwei Zhu,et al.  Robust real-time eye detection and tracking under variable lighting conditions and various face orientations , 2005, Comput. Vis. Image Underst..

[17]  A. Elliot.  Handbook of Approach and Avoidance Motivation , 2008 .

[18]  A. Elliot,et al.  Approach and Avoidance Motivation , 2001 .

[19]  Trevor Darrell,et al.  Adaptive view-based appearance models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Panayiotis G. Georgiou,et al.  Multimodal detection of salient behaviors of approach-avoidance in dyadic interactions , 2012, ICMI '12.

[21]  Marek P. Michalowski,et al.  A spatial model of engagement for a social robot , 2006, 9th IEEE International Workshop on Advanced Motion Control, 2006..

[22]  Rosalind W. Picard,et al.  Automated Posture Analysis for Detecting Learner's Interest Level , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[23]  Brian P. Meier,et al.  Color in Context: Psychological Context Moderates the Influence of Red on Approach- and Avoidance-Motivated Behavior , 2012, PloS one.

[24]  Cecil R. Reynolds,et al.  Revised Children's Manifest Anxiety Scale (RCMAS). Manual , 1985 .

[25]  J. Cacioppo,et al.  Rudimentary determinants of attitudes. II: Arm flexion and extension have differential effects on attitudes. , 1993, Journal of personality and social psychology.

[26]  Kostas Karpouzis,et al.  Feature Extraction and Selection for Inferring User Engagement in an HCI Environment , 2009, HCI.

[27]  P. Beek,et al.  Walk to me when I smile, step back when I’m angry: emotional faces modulate whole-body approach–avoidance behaviors , 2011, Experimental Brain Research.

[28]  Ana Paiva,et al.  Detecting user engagement with a robot companion using task and social interaction-based features , 2009, ICMI-MLMI '09.

[29]  Ana Paiva,et al.  Automatic analysis of affective postures and body motion to detect engagement with a game companion , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[30]  S. Chaiken,et al.  Personality and Social Psychology Bulle- Tin Chen, Bargh / Consequences of Automatic Evaluation Immediate Behavioral Predispositions to Approach or Avoid the Stimulus , 2022 .

[31]  P. Lang.  Cognition in emotion: Concept and action. , 1985 .

[32]  R. Lazarus.  Emotion and Adaptation , 1991 .

[33]  H. Rutherford,et al.  Thriving and Surviving: Approach and Avoidance Motivation and Lateralization , 2011 .

[34]  T. Millon,et al.  Personality and social psychology , 2003 .

[35]  N. Frijda.  The laws of emotion. , 1988, The American psychologist.

[36]  F. Strack,et al.  Movement direction or change in distance? Self- and object-related approach–avoidance motions ☆ , 2008 .

[37]  Pravin Yannawar,et al.  A Review on Speech Recognition Technique , 2010 .

[38]  Lauralee Alben,et al.  Quality of experience: defining the criteria for effective interaction design , 1996, INTR.