Mutual Stance Building in Dyad of Virtual Agents: Smile Alignment and Synchronisation

When we consider communication, the "interactivenature of dialog supports interactive alignment of linguistic representations" [1]. However, verbal communication cannot be reduced to speech. Non-verbal behaviours (NVBs) of interactantsare also taking part in interactive alignment. Based on NVBs alignment, this paper proposes a model of mutual building of stance between virtual agents. In this model, we focus on the the social signal of smile. Indeed, smiles are particular non-verbal markers of agents' stances: their characteristics in terms of facial and dynamical patterns leads to discriminate stances as for instance politeness or amusement. We propose a model combining alignment of types of smile and alignement of timing of smiles (synchronisation). This model enables a dyad of agents to mutually build a common stance, to align on a shared social stance.

[1]  A. Kendon.  Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[2]  Elisabetta Bevacqua.  Positive influence of smile backchannels in ECAs , 2010 .

[3]  D. Biber,et al.  Styles of stance in English: Lexical and grammatical marking of evidentiality and affect , 1989 .

[4]  M. LaFrance.  Nonverbal synchrony and rapport: Analysis by the cross-lag panel technique. , 1979 .

[5]  Jan-Ola Östman,et al.  Handbook of Pragmatics , 2018, Handbook of Pragmatics.

[6]  V. Yngve.  On getting a word in edgewise , 1970 .

[7]  Claire Hughes,et al.  Early Social Cognition , 2010 .

[8]  J. Nadel,et al.  Human responses to an expressive robot , 2006 .

[9]  Catherine Pelachaud,et al.  Effect of time delays on agents' interaction dynamics , 2011, AAMAS.

[10]  Radoslaw Niewiadomski,et al.  How a Virtual Agent Should Smile? - Morphological and Dynamic Characteristics of Virtual Agent's Smiles , 2010, IVA.

[11]  W. S. Condon,et al.  SOUND FILM ANALYSIS OF NORMAL AND PATHOLOGICAL BEHAVIOR PATTERNS , 1966, The Journal of nervous and mental disease.

[12]  Philippe Gaussier,et al.  How an Agent Can Detect and Use Synchrony Parameter of Its Own Interaction with a Human? , 2009, COST 2102 Training School.

[13]  Frank J. Bernieri,et al.  Coordinated movement and rapport in teacher-student interactions , 1988 .

[14]  Brian Vaughan,et al.  Prosodic Synchrony in Co-Operative Task-Based Dialogues: A Measure of Agreement and Disagreement , 2011, INTERSPEECH.

[15]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[16]  Michał B. Paradowski,et al.  The Embodied Language: Why Language Should Not Be Conceived of in Abstraction from the Brain and Body, and the Consequences for Robotics , 2011 .

[17]  Athanasios Katsamanis,et al.  Automatic classification of married couples' behavior using audio features , 2010, INTERSPEECH.

[18]  Robert Englebretson,et al.  Stancetaking in discourse : subjectivity, evaluation, interaction , 2007 .

[19]  Ken Prepin,et al.  Human–machine interaction as a model of machine–machine interaction: how to make machines interact as humans do , 2007, Adv. Robotics.

[20]  Jacqueline Nadel,et al.  Reading sadness beyond human faces , 2010, Brain Research.

[21]  Catherine Pelachaud,et al.  Emotional Meaning and Expression in Animated Faces , 1999, IWAI.

[22]  Radoslaw Niewiadomski,et al.  Smiling virtual agent in social context , 2012, Cognitive Processing.

[23]  Louis-Philippe Morency,et al.  Virtual Rapport 2.0 , 2011, IVA.

[24]  Catherine Pelachaud,et al.  Shared Understanding and Synchrony Emergence - Synchrony as an Indice of the Exchange of Meaning between Dialog Partners , 2011, ICAART.

[25]  J. Cohn,et al.  All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous , 2009, Journal of nonverbal behavior.

[26]  D. Matsumoto,et al.  Spontaneous facial expressions of emotion of congenitally and noncongenitally blind individuals. , 2009, Journal of personality and social psychology.

[27]  Alan Cienki,et al.  Some Uses of Head Tilts and Shoulder Shrugs during Human Interaction, and Their Relation to Stancetaking , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[28]  Laurence R. Horn,et al.  The handbook of pragmatics , 2004 .

[29]  T. Brazelton,et al.  The infant's response to entrapment between contradictory messages in face-to-face interaction. , 1978, Journal of the American Academy of Child Psychiatry.

[30]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .

[31]  M. Auvray,et al.  Perceptual interactions in a minimalist virtual environment , 2009 .

[32]  Elisabeth André,et al.  Catch me if you can: exploring lying agents in social settings , 2005, AAMAS '05.

[33]  E. Hatfield,et al.  Emotional Contagion , 1995 .

[34]  J. Nadel,et al.  Human brain spots emotion in non humanoid robots. , 2011, Social cognitive and affective neuroscience.

[35]  Stacy Marsella,et al.  Virtual Rapport , 2006, IVA.

[36]  Catherine Pelachaud,et al.  Live generation of interactive non-verbal behaviours , 2012, AAMAS.

[37]  A. Manstead,et al.  Can Duchenne smiles be feigned? New evidence on felt and false smiles. , 2009, Emotion.

[38]  Bernard Rimé,et al.  Fundamentals of nonverbal behavior , 1991 .

[39]  M. Pickering,et al.  Why is conversation so easy? , 2004, Trends in Cognitive Sciences.