A Bayesian Theory of Mind Approach to Nonverbal Communication

This paper defines a dual computational framework to nonverbal communication for human-robot interactions. We use a Bayesian Theory of Mind approach to model dyadic storytelling interactions where the storyteller and the listener have distinct roles. The role of storytellers is to influence and infer the attentive state of listeners using speaker cues, and we computationally model this as a POMDP planning problem. The role of listeners is to convey attentiveness by influencing perceptions through listener responses, which we computational model as a DBN with a myopic policy. Through a comparison of state estimators trained on human-human interaction data, we validate our storyteller model by demonstrating how it outperforms current approaches to attention recognition. Then through a human-subjects experiment where children told stories to robots, we demonstrate that a social robot using our listener model more effectively communicates attention compared to alternative approaches based on signaling.

[1]  A. Kendon Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[2]  A. Dittmann Developmental Factors in Conversational Behavior , 1972 .

[3]  Donald W. Fiske,et al.  Face-to-face interaction: Research, methods, and theory , 1977 .

[4]  E. Schegloff Discourse as an interactional achievement : Some uses of "Uh huh" and other things that come between sentences , 1982 .

[5]  D. C. Dermett,et al.  True believers: the intentional strategy and why it works , 1987 .

[6]  Lucille J. Hess,et al.  Acquisition of back channel listener responses to adequate messages , 1988 .

[7]  S. Maynard Analyzing interactional management in native/non-native English conversation : A case of listener response , 1997 .

[8]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[9]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[10]  Nigel G. Ward,et al.  Prosodic features which cue back-channel responses in English and Japanese , 2000 .

[11]  N. Mishima,et al.  The Development of a Questionnaire to Assess the Attitude of Active Listening , 2000 .

[12]  Kristinn R. Thórisson,et al.  Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action , 2002 .

[13]  G. Csibra Teleological and referential understanding of action in infancy. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[14]  J. Bailenson,et al.  Digital Chameleons , 2005, Psychological science.

[15]  Kathryn Robertson,et al.  Active listening: more than just paying attention. , 2005, Australian family physician.

[16]  Ning Wang,et al.  Creating Rapport with Virtual Agents , 2007, IVA.

[17]  K. Otsuka,et al.  Automatic interface of cross-modal nonverbal interactions in multiparty conversation , 2007 .

[18]  Louis-Philippe Morency,et al.  A probabilistic multimodal approach for predicting listener backchannels , 2009, Autonomous Agents and Multi-Agent Systems.

[19]  Julia Hirschberg,et al.  Backchannel-inviting cues in task-oriented dialogue , 2009, INTERSPEECH.

[20]  Thierry Dutoit,et al.  Generating Robot/Agent backchannels during a storytelling experiment , 2009, 2009 IEEE International Conference on Robotics and Automation.

[21]  Dana Kulic,et al.  Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots , 2009, Int. J. Soc. Robotics.

[22]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[23]  Björn W. Schuller,et al.  Building Autonomous Sensitive Artificial Listeners , 2012, IEEE Transactions on Affective Computing.

[24]  Takaki Makino,et al.  Apprenticeship Learning for Model Parameters of Partially Observable Environments , 2012, ICML.

[25]  Raymond H. Cuijpers,et al.  Imitating Human Emotions with Artificial Facial Expressions , 2013, Int. J. Soc. Robotics.

[26]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  S. Marsella,et al.  Social Emotions in Nature and Artifact , 2013 .

[28]  Zhou Yu,et al.  Automatic Prediction of Friendship via Multi-model Dyadic Features , 2013, SIGDIAL Conference.

[29]  Cristina P. Santos,et al.  Facial Expressions and Gestures to Convey Emotions with a Humanoid Robot , 2013, ICSR.

[30]  Cynthia Breazeal,et al.  Computationally modeling interpersonal trust , 2013, Front. Psychol..

[31]  Cynthia Breazeal,et al.  Manipulating Mental States Through Physical Action , 2014, Int. J. Soc. Robotics.

[32]  Chris L. Baker,et al.  Modeling Human Plan Recognition Using Bayesian Theory of Mind , 2014 .

[33]  Sigurđur Örn Ađalgeirsson Mind-theoretic planning for social robots , 2014 .

[34]  Sidney K. D'Mello,et al.  A Review and Meta-Analysis of Multimodal Affect Detection Systems , 2015, ACM Comput. Surv..

[35]  Bilge Mutlu,et al.  Cognitive Human-Robot Interaction , 2016, Springer Handbook of Robotics, 2nd Ed..

[36]  Guy Hoffman,et al.  Computational Human-Robot Interaction , 2016, Found. Trends Robotics.

[37]  Daniel McDuff,et al.  Understanding and Predicting Bonding in Conversations Using Thin Slices of Facial Expressions and Body Language , 2016, IVA.

[38]  Malte F. Jung Affective Grounding in Human-Robot Interaction , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[39]  Cynthia Breazeal,et al.  Role of Speaker Cues in Attention Inference , 2017, Front. Robot. AI.

[40]  Mirko Gelsomini,et al.  Telling Stories to Robots: The Effect of Backchanneling on a Child's Storytelling * , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[41]  Torrin M. Liddell,et al.  Analyzing Ordinal Data with Metric Models: What Could Possibly Go Wrong? , 2017, Journal of Experimental Social Psychology.

[42]  Cynthia Breazeal,et al.  P2PSTORY: Dataset of Children as Storytellers and Listeners in Peer-to-Peer Interactions , 2018, CHI.

[43]  J. Kruschke Rejecting or Accepting Parameter Values in Bayesian Estimation , 2018 .