Nonverbal leakage in robots: Communication of intentions through seemingly unintentional behavior

Human communication involves a number of nonverbal cues that are seemingly unintentional, unconscious, and automatic-both in their production and perception-and convey rich information on the emotional state and intentions of an individual. One family of such cues is called “nonverbal leakage.” In this paper, we explore whether people can read nonverbal leakage cues-particularly gaze cues-in humanlike robots and make inferences on robots' intentions, and whether the physical design of the robot affects these inferences. We designed a gaze cue for Geminoid-a highly humanlike android-and Robovie-a robot with stylized, abstract humanlike features-that allowed the robots to “leak” information on what they might have in mind. In a controlled laboratory experiment, we asked participants to play a game of guessing with either of the robots and evaluated how the gaze cue affected participants' task performance. We found that the gaze cue did, in fact, lead to better performance, from which we infer that the cue led to attributions of mental states and intentionality. Our results have implications for robot design, particularly for designing expression of intentionality, and for our understanding of how people respond to human social cues when they are enacted by robots.

[1]  Paolo Dario,et al.  Effective emotional expressions with expression humanoid robot WE-4RII: integration of humanoid robot hand RCH-1 , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[2]  Scott S. Snibbe,et al.  Experiences with Sparky, a Social Robot , 2002 .

[3]  John A. Johnson,et al.  The international personality item pool and the future of public-domain personality measures ☆ , 2006 .

[4]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[5]  N. Emery,et al.  The eyes have it: the neuroethology, function and evolution of social gaze , 2000, Neuroscience & Biobehavioral Reviews.

[6]  C. Bartneck,et al.  In your face, robot! The influence of a character's embodiment on how users perceive its emotional expressions , 2004 .

[7]  Robert S. Feldman,et al.  Nonverbal Cues as Indicators of Verbal Dissembling1 , 1978 .

[8]  Frank Biocca,et al.  The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments , 2003, Presence: Teleoperators & Virtual Environments.

[9]  Cynthia Breazeal Emotive qualities in robot speech , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[10]  A. Freire,et al.  Are eyes windows to a deceiver's soul? Children's use of another's eye gaze cues in a deceptive situation. , 2004, Developmental psychology.

[11]  V. Csányi,et al.  Intentional behaviour in dog-human communication: an experimental analysis of “showing” behaviour in the dog , 2000, Animal Cognition.

[12]  R. Kraut,et al.  Behavioral roots of person perception: The deception judgments of customs inspectors and laymen. , 1980 .

[13]  G. Hemsley,et al.  The effect of looking behavior on perceptions of a communicator's credibility , 1978 .

[14]  M Miyaji [STOMATITIS AND PHYSIQUE IN 3-YEAR-OLD CHILDREN AND FOOD INTAKE]. , 1964 .

[15]  James F. Roiger,et al.  Testing Interpersonal Deception Theory: The Language of Interpersonal Deception , 1996 .

[16]  Illah R. Nourbakhsh,et al.  The role of expressiveness and attention in human-robot interaction , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[17]  J. Bailenson,et al.  Detecting Digital Chameleons 1 Detecting Digital Chameleons , 2007 .

[18]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[19]  S. Tipper,et al.  Predictive Gaze Cues and Personality Judgments , 2006, Psychological science.

[20]  James J. Lindsay,et al.  Cues to deception. , 2003, Psychological bulletin.

[21]  P. Waxer,et al.  Nonverbal cues for anxiety: an examination of emotional leakage. , 1977, Journal of abnormal psychology.

[22]  S. Baron-Cohen Mindblindness: An Essay on Autism and Theory of Mind , 1997 .

[23]  R. Krauss,et al.  Verbal, vocal, and visible factors in judgments of another's affect. , 1981 .

[24]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[25]  Tetsuo Ono,et al.  Body Movement Analysis of Human-Robot Interaction , 2003, IJCAI.

[26]  Takayuki Kanda,et al.  Three-layered draw-attention model for humanoid robots with gestures and verbal cues , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  M. Zuckerman Verbal and nonverbal communication of deception , 1981 .

[28]  Pamela J. Hinds,et al.  Whose job is it anyway? a study of human-robot interaction in a collaborative task , 2004 .

[29]  Evian Gordon,et al.  In search of the "Duchenne Smile": Evidence from eye movements. , 2001 .

[30]  U. Castiello Understanding other people's actions: intention and attention. , 2003, Journal of experimental psychology. Human perception and performance.

[31]  Robert M. Krauss,et al.  Gesture and Speech in Spontaneous and Rehearsed Narratives , 1994 .

[32]  J. Decety,et al.  From the perception of action to the understanding of intention , 2001, Nature reviews. Neuroscience.

[33]  T. Chartrand,et al.  The chameleon effect: the perception-behavior link and social interaction. , 1999, Journal of personality and social psychology.

[34]  D. Perrett,et al.  Understanding the intentions of others from visual signals: Neurophysiological evidence. , 1994 .

[35]  A. Young,et al.  Reading the mind from eye gaze , 2002, Neuropsychologia.

[36]  Raj M. Ratwani,et al.  Integrating vision and audition within a cognitive architecture to track conversations , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[37]  Luca Turella,et al.  When Gaze Turns into Grasp , 2006, Journal of Cognitive Neuroscience.

[38]  P. Ekman,et al.  The Duchenne smile: emotional expression and brain physiology. II. , 1990, Journal of personality and social psychology.

[39]  Aaron Powers,et al.  Matching robot appearance and behavior to tasks to improve human-robot cooperation , 2003, The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003. Proceedings. ROMAN 2003..

[40]  A. Leslie Pretense and representation: The origins of "theory of mind." , 1987 .

[41]  Tetsunori Kobayashi,et al.  A conversational robot utilizing facial and body expressions , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[42]  T. Kanda,et al.  Robot mediated round table: Analysis of the effect of robot's gaze , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[43]  C. Kleinke Gaze and eye contact: a research review. , 1986, Psychological bulletin.

[44]  Candace L. Sidner,et al.  Where to look: a study of human-robot engagement , 2004, IUI '04.

[45]  Michael Argyle,et al.  The communication of friendly and hostile attitudes by verbal and non‐verbal signals , 1971 .

[46]  S. Tipper,et al.  Gaze cueing of attention: visual attention, social cognition, and individual differences. , 2007, Psychological bulletin.

[47]  E. Goffman Relations in Public: Microstudies of the Public Order , 1971 .

[48]  Lee Sproull,et al.  Cooperating with life-like interface agents , 1999 .

[49]  K. Scherer,et al.  Minimal cues in the vocal communication of affect: Judging emotions from content-masked speech , 1972, Journal of psycholinguistic research.

[50]  Bilge Mutlu,et al.  A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[51]  Reid G. Simmons,et al.  Interactions with a moody robot , 2006, HRI '06.

[52]  P. Ekman,et al.  Nonverbal leakage and clues to deception. , 1969, Psychiatry.

[53]  Kristine L. Nowak Defining and Differentiating Copresence, Social Presence and Presence as Transportation , 2001 .

[54]  Brian Scassellati,et al.  How to build robots that make friends and influence people , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[55]  Sara B. Kiesler,et al.  Mental models of robotic assistants , 2002, CHI Extended Abstracts.

[56]  Klaus R. Scherer,et al.  The voice of confidence: Paralinguistic cues and audience evaluation. , 1973 .

[57]  Tetsuo Ono,et al.  Physical relation and expression: joint attention for human-robot interaction , 2003, IEEE Trans. Ind. Electron..

[58]  C. Frith,et al.  Autism, Asperger syndrome and brain mechanisms for the attribution of mental states to animated shapes. , 2002, Brain : a journal of neurology.

[59]  J. Hietanen,et al.  Facial and emotional reactions to Duchenne and non-Duchenne smiles. , 1998, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.