Automatic Prediction of Friendship via Multi-model Dyadic Features

In this paper we focus on modeling friendships between humans as a way of working towards technology that can initiate and sustain a lifelong relationship with users. We do this by predicting friendship status in a dyad using a set of automatically harvested verbal and nonverbal features from videos of the interaction of students in a peer tutoring study. We propose a new computational model used to model friendship status in our data, based on a group sparse model (GSM) with L2,1 norm which is designed to accommodate the sparse and noisy properties of the multi-channel features. Our GSM model achieved the best overall performance compared to a non-sparse linear model (NLM) and a regular sparse linear model (SLM), as well as outperforming human raters. Dyadic features, such as number and length of conversational turns and mutual gaze, in addition to low level features such as F0 and gaze at task, were found to be good predictors of friendship status.

[1]  Catherine Pelachaud,et al.  Mutual Stance Building in Dyad of Virtual Agents: Smile Alignment and Synchronisation , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Alastair J. Gill,et al.  Coordination in Conversation and Rapport , 2007 .

[4]  Justine Cassell,et al.  Negotiated Collusion: Modeling Social Language and its Relationship Effects in Intelligent Agents , 2003, User Modeling and User-Adapted Interaction.

[5]  M. Azmitia,et al.  Friendship, transactive dialogues, and the development of scientific reasoning , 1993 .

[6]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[7]  Nigel Ward,et al.  Non-lexical conversational sounds in American English , 2006 .

[8]  J. Cassell,et al.  Nudge nudge wink wink: elements of face-to-face conversation for embodied conversational agents , 2001 .

[9]  Harry T. Reis,et al.  Stability and Change in Relationships: Contributors , 2002 .

[10]  R. Rosenthal.  Sensitivity to Nonverbal Communication: The PONS Test , 1979 .

[11]  B. Berg.  Qualitative Research Methods for the Social Sciences , 1989 .

[12]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[13]  P. Liamputtong.  Qualitative Research Methods , 2005 .

[14]  Timothy W. Bickmore,et al.  Designing Relational Agents as Long Term Social Companions for Older Adults , 2012, IVA.

[15]  Timothy W. Bickmore,et al.  Establishing and maintaining long-term human-computer relationships , 2005, TCHI.

[16]  M. Argyle,et al.  EYE-CONTACT, DISTANCE AND AFFILIATION. , 1965, Sociometry.

[17]  Kohji Fukunaga,et al.  Introduction to Statistical Pattern Recognition-Second Edition , 1990 .

[18]  Jieping Ye,et al.  Multi-Task Feature Learning Via Efficient l2, 1-Norm Minimization , 2009, UAI.

[19]  Yukiko I. Nakano,et al.  Estimating user's engagement from eye-gaze behaviors in human-agent conversations , 2010, IUI '10.

[20]  Justine Cassell,et al.  Reactive Redundancy and Listener Comprehension in Direction-Giving , 2008, SIGDIAL Workshop.

[21]  Hao Yan,et al.  More than just a pretty face: conversational protocols and the affordances of embodiment , 2001, Knowl. Based Syst..

[22]  Frank J. Bernieri,et al.  Coordinated movement and rapport in teacher-student interactions , 1988 .

[23]  Daniel Hogan,et al.  The 30-Sec Sale: Using Thin-Slice Judgments to Evaluate Sales Effectiveness , 2006 .

[24]  Stacy Marsella,et al.  Virtual Rapport , 2006, IVA.

[25]  H. Timothy Bunnell,et al.  Emotion Identification for Evaluation of Synthesized Emotional Speech , 2012 .

[26]  Catherine Pelachaud,et al.  Rules for Responsive Robots: Using Human Interactions to Build Virtual Interactions , 2002 .

[27]  Ronald E. Riggio,et al.  Effects of Reciprocal Peer Tutoring on Academic Achievement and Psychological Adjustment: A Component Analysis. , 1989 .

[28]  Robert Rosenthal,et al.  Physicians' H]ead and Body Positions as Determinants of Perceived Rapport , 1983 .

[29]  D. Bolinger.  Intonation and its parts : melody in spoken English , 1987 .

[30]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[31]  Dirk Heylen,et al.  Designing for friendship: Becoming friends with your ECA , 2002 .

[32]  Amy Ogan,et al.  Rudeness and Rapport: Insults and Learning Gains in Peer Tutoring , 2012, ITS.

[33]  Yukiko I. Nakano,et al.  Towards a Model of Face-to-Face Grounding , 2003, ACL.

[34]  Maja Pantic,et al.  Social signal processing: Survey of an emerging domain , 2009, Image Vis. Comput..

[35]  Jerold L. Hale,et al.  Nonverbal expectancy violations: Model elaboration and application to immediacy behaviors , 1988 .

[36]  J. Youniss.  Parents and Peers in Social Development: A Sullivan-Piaget Perspective , 1980 .

[37]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[38]  William Yang Wang,et al.  “Love ya, jerkface”: Using Sparse Log-Linear Models to Build Positive and Impolite Relationships with Teens , 2012, SIGDIAL Conference.

[39]  Elmar Nöth,et al.  The INTERSPEECH 2012 Speaker Trait Challenge , 2012, INTERSPEECH.

[40]  L. Tickle-Degnen,et al.  The Nature of Rapport and Its Nonverbal Correlates , 1990 .

[41]  Mark S. Granovetter.  The Strength of Weak Ties , 1973, American Journal of Sociology.

[42]  Timothy W. Bickmore,et al.  Relational Agents Improve Engagement and Learning in Science Museum Visitors , 2011, IVA.

[43]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[44]  Xi Chen,et al.  Smoothing proximal gradient method for general structured sparse regression , 2010, The Annals of Applied Statistics.

[45]  R. Rosenthal,et al.  Rapport expressed through nonverbal behavior , 1985 .

[46]  Daniel Gatica-Perez,et al.  Automatic nonverbal analysis of social interaction in small groups: A review , 2009, Image Vis. Comput..